The Page Cache and the Writeback Path

write returns a count, and the count means the bytes were accepted, not that they were stored. Between the system call and the platter there is a cache of file pages shared by every process, a set of dirty bits, and a background thread that decides when to flush. Understanding that path is what makes it possible to reason about durability, about why fsync is expensive, and about why a crash can lose writes that a program believed it had made.

Pages, not buffers

The page cache indexes file data by page: a mapping from an inode and a page offset to a physical page frame holding that file’s bytes. A read of a page that is not present allocates a frame, asks the filesystem to populate it, and returns the data; a read of a page that is present copies out of the frame without touching the device. A write to a page that is present copies into the frame and marks it dirty; a write to a page that is not present allocates the frame first, and the newly allocated page is filled from disk before the write is applied if the write covers only part of it.

The last condition is the read-modify-write that partial writes induce. Writing 100 bytes into a page that was never read requires the page’s other 3996 bytes to be correct, and the only source of correctness is the device, so the kernel reads the page first. A write that covers the whole page can skip the read, which is why sequential writes of block-aligned buffers avoid half the device traffic of scattered small writes.

The unit of caching, of dirty tracking, and of writeback is the page, typically 4096 bytes, which is coarser than the block size most filesystems use. A page that holds four 1 KiB blocks carries a buffer head for each of them, so the filesystem can allocate and write back a single block without touching its neighbours; where the block size equals the page size the two units coincide and that per-block bookkeeping disappears.

/* Schematic: the three steps between a write() and a dirty page.
 * Not kernel source. */
ssize_t buffered_write(struct file *f, const void *ubuf, size_t n, loff_t *pos)
{
        /* 1. copy user bytes into pages, allocating and reading as needed */
        copy_into_pages(f->mapping, ubuf, *pos, n);
        /* 2. mark the touched pages dirty in the inode's page index */
        mark_dirty(f->mapping, *pos, n);
        /* 3. no device I/O here at all; the flusher writes later */
        return n;
}

Dirty tracking and the thresholds

A page marked dirty is added to the inode’s dirty list and the inode to a global list of dirty inodes. Nothing is written immediately. The kernel maintains two watermarks as fractions of total memory: a background threshold at which a flusher thread starts writing dirty pages, and a limit at which a process that dirties another page is forced to write some itself. The first is a smoothness mechanism, the second is back-pressure that prevents a writer from consuming all memory with unflushed data.

Between those thresholds, dirty data accumulates. A program that writes a hundred megabytes on a machine with several gigabytes of memory has, at exit, written almost nothing to the device; the close returns immediately and the pages remain dirty. That is not a bug and not a policy failure: the cache exists so that a burst of writes can be reordered and coalesced by the device rather than issued in the order the program produced them.

The flush decision is per-inode and per-page. A flusher thread picks dirty inodes in a rotating order, writes their dirty pages in increasing offset order within each inode, and skips pages that are locked by another writer. Writing in offset order is what makes the device see sequential traffic even when the program wrote scattered pages, and it is why a scattered write pattern can still flush as a run of adjacent requests.

fsync and the two things it must do

fsync asks for two separate guarantees: the file’s data must reach the device, and the metadata needed to find that data must also reach the device. The second is easy to overlook. A newly allocated block is referenced by a block map or an extent record, and if the data block reaches the device but the record naming it does not, the file after a crash has the data on disk but no way to reach it.

The ordering requirement follows from that. A filesystem that writes data blocks before the transaction that records their allocation is safe against a crash between the two, because the transaction is simply discarded: the bitmap still marks the blocks free, the inode still has its old map, and the write is lost but the volume is consistent. A filesystem that commits the metadata first is not safe: a crash leaves a record pointing at a block that was never written, and the file reads stale content. This is the same ordering constraint that governs Delayed Allocation and the Reservation Window, where the allocation decision is itself deferred to writeback.

fsync on a file does not flush the containing directory’s metadata. Creating a file dirties the directory’s data block and its inode; fsync on the new file’s descriptor does not touch either. A program that creates a file, writes it, calls fsync on it, and returns is not crash-safe, because after a crash the name may not exist. The correct sequence flushes the file and then the directory.

int fd = open("out.dat", O_WRONLY | O_CREAT, 0644);
write(fd, buf, len);
fsync(fd);                       /* data and the file's own metadata */
close(fd);
int dfd = open(".", O_RDONLY | O_DIRECTORY);
fsync(dfd);                      /* the name's existence, in the directory */

The two calls cost what they cost. Each is a request to the device to make its write cache durable, and on a device with a volatile write cache that request is a cache-flush command, which is not free. fdatasync is the cheaper variant: it flushes data and the metadata strictly required to retrieve it, and may skip a modification-time update that does not affect retrieval.

mmap and the same pages

A memory mapping of a file is not a separate data path. The mapping installs page-table entries pointing at the same page cache frames, so a store through the mapping dirties the same page that write would have dirtied. There is no copy and no separate buffer. The difference is that the store is not a system call: the page may be marked dirty by the processor’s write to memory, and the kernel discovers the dirty page by scanning page tables or by write-protecting pages and taking a fault.

Dirty accounting for mapped pages is coarser than for written pages, because the hardware does not tell the kernel which bytes changed: a store dirties the page, and the kernel keeps no record of the byte range within it. Writeback therefore treats the whole page as modified, and a filesystem whose blocks are smaller than a page writes back every mapped block in it, so one store through a mapping can push several blocks to the device.

msync with the synchronous flag is the mapped equivalent of fsync: it writes the dirty pages in the range and waits for them. Without it, unmapping a file does not force a write; the pages stay dirty in the cache and are flushed on the ordinary schedule.

Failure modes at the boundary

Three failures are worth naming. The first is silent loss on write success followed by a crash before writeback; nothing in the interface promises otherwise, and programs that need durability must ask for it. The second is delayed allocation failure: a write may be accepted into the cache when no space is available, and the failure surfaces later, at writeback, where the writing process is no longer present to receive it. Filesystems that reserve space at write time avoid this, at the cost of doing allocation work in the write path.

The third is the interaction with the filesystem’s own consistency mechanism. If a crash occurs while writeback is in progress, some pages are on the device and some are not, and the metadata may be in any state. The journal is what makes the resulting volume recoverable, which is why the writeback order and the journal commit order must agree: the pages covered by a transaction must reach their home blocks before that transaction commits, or recovery can leave a file whose map names blocks that were never written. The mount path that runs that recovery is described in Mounting: From Device Node to a Live Superblock.

The page cache is therefore not a separate optimisation layer that a filesystem can opt out of casually. It is the place where the interface’s promises about durability are actually kept or broken, and the filesystem’s job is to make sure that when the cache decides to write, the order it writes in is one that recovery can understand.