Short Counts: Why read and write Return Less Than Asked

The read and write system calls take a byte count and return a byte count. The two numbers are frequently different, and the difference is not an error. A return value of 800 from a read of 4096 bytes means 800 bytes were transferred, the call succeeded, and the remaining 3296 bytes were simply not delivered. Code that assumes the request size and the result size agree will work on a regular file with a warm page cache and fail on a terminal, a pipe, or a socket, often months after it was written.

What the standard actually guarantees

POSIX is explicit that the return value is the number of bytes actually transferred, and that this may be less than requested. The specification describes the conditions under which a short transfer can occur rather than promising a full one. For a regular file, the shortfall is bounded by the remaining file size: reading 4096 bytes at an offset four bytes before end-of-file returns 4, and the next call returns 0.

Zero is a distinct result from a short read. For a regular file it means end-of-file. For a socket, stream or datagram, it means the peer performed an orderly shutdown of its sending direction. For a terminal, it is unusual. These are different events in the program’s state machine, and collapsing them into “no data available” is a common source of loops that never terminate: a loop that retries on zero reads spins forever at end-of-file.

Errors and short counts are also distinct. EINTR means a signal arrived before any byte was transferred, so the call can be retried as is. EAGAIN on a non-blocking descriptor means nothing was available. A short count with no error means some bytes were transferred and the rest were not, and the offset has advanced by the short count.

The cases, by object type

The reasons for a partial transfer depend entirely on what the descriptor refers to.

A regular file on disk returns the full count unless it hits end-of-file or a resource limit. The exceptions are worth knowing: a read that crosses into a hole in a sparse file returns zeros rather than short data; a read interrupted by a signal after some bytes were transferred returns the partial count rather than EINTR; and a request larger than the kernel’s maximum transfer size is truncated to that size. Linux bounds a single call at MAX_RW_COUNT, derived from INT_MAX and just under 2 GiB, so a request above it is shortened by the kernel rather than by the device. POSIX requires that a count above SSIZE_MAX produce an implementation-defined result, which in practice means the same truncation.

A pipe returns at most what is currently in the buffer. The default pipe capacity is 64 KiB on Linux, and the buffer is filled by the writer and drained by the reader, so a read of 4096 bytes when only 100 are buffered returns 100. Pipes also have a guaranteed atomicity property: a write of up to PIPE_BUF bytes, 4096 on Linux and 512 as the POSIX minimum, is not interleaved with writes from other processes. Above that size, writes may be split.

A stream socket returns whatever has arrived in the receive queue, up to the requested count. There is no relation between the boundaries the peer used when sending and the boundaries the reader sees. Two writes of 100 bytes may be delivered as one read of 200, and one write of 2000 bytes may arrive as reads of 1400 and 600. This is the defining property of a byte-stream protocol, and any code that treats read boundaries as message boundaries is broken regardless of how the network behaves in testing.

A terminal in canonical mode returns at most one line, ending at the newline, regardless of the count requested. A read of 1024 bytes on a terminal where the user typed four characters and pressed enter returns 5, including the newline.

/* Read exactly n bytes, or report how many were obtained. */
#include <errno.h>
#include <unistd.h>

ssize_t read_full(int fd, void *buf, size_t n) {
    size_t got = 0;
    char *p = buf;

    while (got < n) {
        ssize_t r = read(fd, p + got, n - got);
        if (r < 0) {
            if (errno == EINTR) continue;      /* nothing transferred */
            return -1;
        }
        if (r == 0) break;                     /* EOF or peer shutdown */
        got += (size_t)r;
    }
    return (ssize_t)got;
}

The distinction between returning 0 and returning -1 matters more than the loop itself. A framing protocol that expects a fixed-size header needs to know whether the peer closed cleanly between two headers, which is an ordinary end of stream, or closed in the middle of one, which is a truncated message and usually a protocol error.

The write side is stricter

write has a stronger contract than read for blocking descriptors on regular files: POSIX requires that a write to a regular file either transfer all the bytes or fail, with the exception of interruption. The same is not true for pipes and sockets. A write to a pipe larger than the free space blocks until it can complete, and on a non-blocking pipe it transfers what fits and returns that count. A write to a socket may transfer part of the buffer when the send queue fills, and the remainder must be written later.

Partial writes are where the naive loop fails most visibly, because a program that writes a header and a payload in two calls and ignores the return of the first has written neither correctly. The loop is the mirror of the read loop, with one addition: it must handle the case where the descriptor is non-blocking and the return is EAGAIN after some bytes went out. Retrying immediately then spins; the correct action is to wait for writability, which is what the interfaces in readiness notification provide.

writev and readv reduce the number of calls but do not change the contract. writev takes an array of buffers and returns the total bytes written, which may still be short and may stop in the middle of an element. The count is a byte total, not an element count, so the caller must walk the array to find where the transfer stopped. The same is true of sendmsg with multiple elements.

Buffering hides the shortfall, until it doesn’t

The C library’s buffered I/O functions present a different contract. fread returns the number of items read, and it loops internally until it has the requested count or hits end-of-file or an error, so a caller asking for one 4096-byte item gets either 1 or 0, not a partial item. fwrite similarly loops. This is why code that works through FILE * often breaks when rewritten to use descriptors directly: the buffering layer was absorbing the short counts.

The cost of that absorption is latency and a second copy. Every byte passes through the library buffer, and a program that reads a large file through fread copies each block into user memory twice. The kernel copy is unavoidable; the library copy is not, and it is why high-throughput readers use descriptors with a large buffer of their own.

The buffered layer also changes the meaning of a return of zero. fread returning a short item count means end-of-file or error, with ferror distinguishing them; it never means “try again shortly” for a blocking file. Mixing buffered and unbuffered access on the same descriptor is legal but produces exactly the confusion the layering was meant to prevent, because the library’s read-ahead has already advanced the shared offset described in file descriptors as table indices.

Retrying, restarting, and the signal interaction

Two failure modes deserve separate handling. A return of -1 with errno set to EINTR means the call transferred nothing, so it can be repeated verbatim. A short positive return after a signal means some data moved; the loop must continue from the new offset rather than restart the whole request. Conflating the two either loses data or duplicates it.

Whether a signal produces EINTR at all depends on how the handler was installed. With SA_RESTART, the kernel restarts certain interrupted calls automatically, and read and write on a slow object such as a terminal or socket are among them; the call resumes and the program never sees the interruption. Without the flag, the call returns EINTR. The set of calls that are restartable is not uniform across objects, and the details are treated with signal delivery in signal handling and async-signal safety.

For a non-blocking descriptor the loop needs a third branch. EAGAIN and EWOULDBLOCK are the same value on Linux; POSIX allows them to differ, so portable code tests both. A loop that treats them as fatal errors will abort on a connection that merely had nothing to deliver at that instant.

What the return count does not establish

A completed write says the bytes reached the kernel, not the disk. For a regular file the data sits in the page cache, and only fsync or fdatasync forces it to stable storage. For a socket the bytes reached the local send queue, not the peer. The distinction is not pedantic: a program that writes a record, returns success to its caller, and crashes leaves the record in a state the filesystem may or may not have committed. The boundary between “accepted” and “durable” is drawn by the sync calls, and the same reasoning applies to memory-mapped output, where the store completes in the mapping and the writeback happens later.