Shingled Magnetic Recording and the Write Cursor

A hard drive’s write head is physically wider than its read head, and it has been for most of the technology’s history — the two elements are separate structures on the same slider, as described in the magnetic read and write path. The write head must generate a field strong enough to switch the medium, and field strength trades against the sharpness of the field’s falloff at the pole edges; making the writer narrow enough to match the reader would cost more gradient than the density gain is worth. Shingled recording accepts the wide writer and recovers the density by overlapping the tracks, so that the medium’s track pitch is set by the reader rather than the writer.

Overlapping tracks raise the density

Write a track. Then move the head by less than the writer’s width and write the next one. The second write partially overwrites the first, erasing its trailing edge and leaving a strip whose width is the difference between the writer width and the step. Repeat, and the surface becomes a series of overlapping strips, like roof shingles — hence the name.

What survives of each track is only as wide as the reader needs, so the track pitch is now limited by how narrow the reader can be made and by how well the head can be positioned, not by the writer. The density gain is the ratio of the writer width to the step, minus the space lost to the guard region at the edge of each group of shingled tracks.

The cost is immediate: the medium inside a shingled group can no longer be written in arbitrary order. Writing track n destroys part of track n−1. To rewrite track n−1, the drive must rewrite n−1, then n, then n+1, and so on to the end of the group, because each rewrite damages its predecessor. A single-track random write becomes a rewrite of every track after it in the group.

The group is called a band, and its length is a design parameter. A long band maximises the fraction of the surface that is shingled and therefore the density gain; a short band limits the cost of a random rewrite. Vendors choose a length that balances the two for the intended workload, and the band structure is invisible to a host that does not ask about it.

Three ways to expose the constraint

A drive-managed shingled drive hides the bands entirely. It presents ordinary logical block addresses, maintains a mapping from logical addresses to physical tracks, and absorbs random writes into a conventional-media region used as a persistent cache. Writes land in the cache and are later migrated into the shingled region in sequential order. The host sees a normal disk with a large cache; the drive pays for the migration internally, and the cost surfaces as write amplification, as latency spikes when the cache is being cleaned, and as a capacity that is smaller than the raw shingled area because part of the medium is reserved for the cache.

A host-managed drive exposes the bands as zones and refuses to hide the constraint. Writes must be sequential within a zone, starting at the zone’s write pointer and advancing; a write at an address other than the pointer is rejected. The host is responsible for placing data so that sequential access stays sequential, and for managing its own reclamation.

A host-aware drive sits between the two: it accepts random writes and maintains the mapping, but reports which zones are shingled and which are conventional, so a host that understands the hint can place data well while a host that does not still works.

The standards for the host-managed and host-aware cases are the SCSI Zoned Block Commands and the ATA Zoned Command Set. Both define a device model in which the address space is divided into zones, each with a type, a capacity, and a state.

Zone state and the write pointer

A zone has a starting address, a length, and a write pointer that begins at the start and advances with each write. A zone whose pointer is at its start is empty; one whose pointer has reached the end is full; a zone that has been written but is not full is open. The number of open zones is limited, because each open zone costs the drive bookkeeping and, in some implementations, a reserved band.

Zone types distinguish the constraints. A conventional zone behaves like ordinary disk and can be written in any order. A sequential-write-required zone accepts writes only at the write pointer and rejects anything else with an error. A sequential-write-preferred zone accepts random writes but may perform poorly; it is a hint rather than a rule.

The zone capacity can be smaller than the zone size. The difference is the gap at the end of each band that separates it from the next, and the host must use the capacity rather than the size when deciding how much data fits. A host that fills to the zone size will find the last writes rejected.

The write pointer is also the recovery mechanism. After a power failure, the drive cannot know how much of an in-flight write completed, but it knows the pointer’s value was durable at some earlier point. Reads of a zone beyond the pointer return data the drive does not guarantee, and the host is expected to re-verify what it wrote. This is the zoned analogue of a filesystem journal, and it is why a zoned device’s write ordering is part of its contract rather than an implementation detail.

/* Cost of rewriting one logical track inside a shingled band.
   Rewriting at offset k requires rewriting every track from k to the band end. */
struct band { uint32_t first_track; uint32_t ntracks; uint32_t track_bytes; };

static uint64_t rewrite_cost(const struct band *b, uint32_t track_in_band)
{
    if (track_in_band >= b->ntracks)
        return 0;                                   /* outside the band */
    uint64_t remaining = b->ntracks - track_in_band;
    return remaining * (uint64_t)b->track_bytes;     /* bytes rewritten, one pass */
}

/* A write of one sector costs the whole tail of the band. Amplification is
   the ratio of that tail to the bytes the host asked to change. */
static double band_amplification(const struct band *b, uint32_t track_in_band,
                                 uint32_t sector_bytes)
{
    return (double)rewrite_cost(b, track_in_band) / (double)sector_bytes;
}

The function makes the trade explicit: the amplification is proportional to how far into the band the write lands, so a host that appends to zones in order pays nothing and a host that rewrites the first track of every band pays for the whole band each time. Placement is not an optimisation on this medium; it is the difference between a working device and an unusable one.

Where the model breaks

The failure mode of a host-managed device is a host that treats it as a random-access disk. Every out-of-order write is rejected, and a filesystem that allocates blocks by free-list rather than by append will generate them constantly. Filesystems that support zoned devices therefore change their allocation strategy: metadata goes into conventional zones, data goes into sequential zones appended in allocation order, and garbage collection moves live data between zones in whole-zone units.

That last point is the structural one. A zoned device cannot reclaim a partially live zone in place; the host must copy the live data to a new zone, reset the old one, and update its references. The host’s garbage collector now has the same shape as the flash controller’s, with the same utilisation term in its write amplification, which is why the two technologies converge on similar software despite having nothing in common physically.

The drive-managed case has its own failure mode: sustained random writes fill the conventional cache faster than the drive can migrate, and the migration competes with host traffic for the same actuators. The observable symptom is a latency distribution with a long tail: most writes are served from the cache, and a minority wait behind a band rewrite whose cost is measured in milliseconds. Averages hide this; the tail is where the queueing happens.

There is also a capacity illusion in the drive-managed case. The logical capacity is fixed, but the amount of data the drive can hold in its shingled region depends on how much of it is in the cache, and the drive’s ability to keep the cache drained depends on the write pattern. A drive that is nearly full has less room to migrate, and the migration policy becomes conservative — which shows up as a throughput that degrades as the drive fills, not because the medium is slower but because the reclamation has less free space to work with.

The interaction with the drive’s own cache is the same problem the flash translation layer solves, with the same parameters: how much capacity to withhold, when to clean, and how to order the cleaning against host traffic. The difference is that a shingled drive’s reclamation is mechanical, so its cost is measured in milliseconds per band rather than in microseconds per page, and the queueing behaviour described for conventional drives in seek time and rotational latency applies to the reclamation traffic as well as to the host’s.