Blocking Records on Magnetic Tape and the Cost of the Interblock Gap

A tape drive cannot start and stop instantly. The tape moves past a read/write head at a constant linear speed, and the head can only read or write while the tape is moving at that speed. Between one block of data and the next the transport must decelerate, reverse, and accelerate back to speed, and during that interval nothing can be written. The blank stretch of tape consumed by that manoeuvre is the interblock gap, and its length is a property of the drive rather than of the data.

The gap is the reason tape files have a block size at all. Every write of a block costs one gap, so the fraction of tape holding data is the block length divided by the block length plus the gap length. With a gap of roughly half an inch and blocks of eighty bytes, the tape is almost entirely gap. With blocks of several kilobytes the gap becomes negligible. The same arithmetic governs the transfer rate: each gap forces the transport through a stop and a restart, so a file of tiny blocks spends most of its time accelerating and decelerating rather than transferring.

The arithmetic that forces large blocks

Take a 2,400-foot reel, which is 28,800 inches of tape, and a drive whose gap, including the distance the transport needs to stop and restart, is about 0.6 inch. Payload fraction is block length divided by block length plus gap length, and the length of a block on tape is its byte count divided by the recording density. At the highest nine-track density of 6,250 bits per inch, one inch of tape holds about 780 bytes, so the whole reel holds roughly 22 megabytes with no gaps at all. A block of 4,000 bytes occupies about 5.1 inches and leaves a payload fraction near ninety percent. A block of 80 bytes occupies about 0.1 inch, so the gap is six sevenths of the distance each block consumes and the same reel holds under four megabytes. That is the entire argument for blocking: the same reel holds about six times as much data at 4,000 bytes per block as at 80.

The second half of the cost is time. A drive that stops and restarts for every block spends a fixed overhead per block, independent of block length, on the order of a few milliseconds for the mechanical cycle. A file of forty thousand small blocks therefore pays that overhead forty thousand times. Grouping the same records into a few thousand larger blocks pays it a few thousand times. The instruction count to move the data is identical; the wall-clock difference comes entirely from the mechanics.

Block size is not free, and the constraint that limits it is the buffer. A read of a block requires a buffer of the block’s length in main storage, and the program’s data area is sized for the largest block it will read. On a machine where a job’s region might be a few tens of kilobytes, a 32 KB block leaves very little room for anything else, and a 64 KB block will not fit. The block size is therefore chosen as the largest value that the smallest program which must read the file can accommodate. This is why record layouts in the era are specified with a block size that looks arbitrary: it is the result of a negotiation between tape efficiency and the memory available to the least-endowed reader.

There is a third cost, less obvious, in error recovery. A tape error is detected per block, by a parity check across each frame and often a cyclic redundancy check over the block. When a block fails, the drive repositions and retries the read. A short block means the retry is cheap; a long block means a single bad spot invalidates more data and the retry costs more tape motion. Block sizes that are very large trade recovery granularity for throughput.

Fixed, variable, and spanned blocks

The block is a unit of physical I/O, and the record is a unit of logical data. The relationship between the two is a format decision, and the format has to be recorded somewhere, because the tape carries no self-describing header on each block.

In fixed-length blocked format every block holds an integral number of records, and the reader is told the record length and the block size out of band. Reading then requires only that the count of records per block be computed; the block is chopped into records by arithmetic. In variable-length format each block carries its own length in a four-byte header, so a block can hold records of differing lengths and the reader can parse it without knowing the layout in advance. The variable format pays four bytes per block and requires the reader to trust the length field, which is the classic failure: a corrupted length field makes the reader consume the following block’s header as data.

Spanned format exists for records longer than a block. A record too large for one block is written across several, with an indicator in the block header saying whether the block begins, continues, or ends a record. This is how a single logical record larger than the buffer is handled, and it is why a reader that assumes one record per block fails on such files.

The consequence for a program is that the block size and the record format are not recoverable from the tape. They are attributes that the job control statement supplies, and a wrong value produces either a data error or, worse, a plausible misreading. A file written with fixed blocks of 4,000 bytes holding 80-byte records read with a declared record length of 100 produces records that are shifted relative to the real ones, and every field is garbage without any error being raised. Numeric fields suffer worst, because a misaligned packed decimal field can still look like a plausible number.

Tape marks, and why a gap is not enough

A gap separates blocks, but a gap cannot distinguish the end of one file from the end of the volume or from a blank stretch caused by damage. Tape therefore has a dedicated marker: a tape mark, written as a special block whose pattern is not a valid data block. The drive recognises it and reports it as a distinct condition rather than as a data transfer.

Tape marks nest. A single tape mark ends a file. Two consecutive tape marks end the data on the volume, a convention that lets a reader stop without reading to the physical end of the reel. This nesting is the whole file structure of an unlabelled tape, and it is why a program reading a multi-file tape loops on “read until tape mark” once per file rather than tracking byte offsets.

The marker is also the reason a write of a short block is dangerous in a way that is not obvious. Because a tape mark is a physical pattern, a partially written block interrupted by a power failure can leave a fragment that the drive reads as a short block, and the file’s structure after that point is undefined. Tape writing is therefore done with the convention that a file is not considered complete until the closing tape marks are written, and recovery from a failed write means discarding everything after the last known good mark. This is the same reasoning that leads to checkpoint and restart on the compute side: a durable marker is what makes it possible to resume without trusting everything that came before.

Labels: the metadata the marks cannot carry

An unlabelled tape tells a reader where files begin and end and nothing else. It does not say which data set is on it, when it was written, when it expires, or what block size it uses. Those facts live on a label: a header block at the start of the volume and a header block at the start of each file, written in a fixed format so that a program can read them without knowing anything about the data.

A standard label carries a identifier, a volume serial, a file sequence number, a creation date, and an expiration date, among other fields. The expiration date is the mechanism that protects a tape from being overwritten by a later job: a catalog entry says the data set lives on this volume, and the volume’s label says it may not be reused before a date. A job that asks to write to a volume whose label has not expired is refused unless it explicitly overrides, which is the only thing standing between a misdirected output data set and the destruction of a month’s work.

The label also carries the record format, block size, and record length, so a reader can size its buffer from the header rather than from the job statement. This is the reason labelled tapes are the norm in a shop with more than one program: the layout travels with the data.

The trade is that labels are read and written by the access method, not by the program, and a program that bypasses the access method to write raw blocks will produce a tape whose label claims a format the data does not have. Reading such a tape with the labelled access method yields errors at the first block, because the declared block size does not match the physical one. Unlabelled tapes avoid that mismatch by having no claims at all, which is why they survive in transfers between installations that do not share a catalog.

What replaced it, and what did not

Disk made the gap argument obsolete, and the first use disk was put to in batch was spooling, which replaced unit-record devices with disk data sets precisely because a disk write does not force a stop and restart. A disk seek costs a mechanical delay that is not proportional to the transfer length in the same way, and a disk file does not require every write to be a stop-and-restart cycle. Blocking survives on disk for a different reason: the file system allocates in fixed units, and reading a record at a time through a buffer manager costs a system call per record. The optimal block size on disk is set by the page size and the prefetch policy rather than by a gap.

Tape survived for a long time for the one property disk did not have at a comparable price: a removable medium with a low cost per byte, suited to a copy that is written once and stored. The formats changed, the gap shrank, and the block size grew, but the structure of a stream of blocks ended by a marker persisted. A modern archive format that concatenates records into large blocks and terminates the stream with a distinct end marker is doing what the tape mark did, for the same reason: the reader needs a terminator that no data block can imitate.