procfs and the Limits of Representing Kernel State as Files

The argument that a uniform file interface can stand in for every other interface is old and was made most forcefully by the systems that followed Unix rather than by Unix itself. procfs is the place where the argument is tested, because it presents live kernel state through the file operations and the fit is imperfect in instructive ways. The files exist, they have names, and read returns text. They also have no stable size, no meaningful offset for some of them, and content that can change between two reads of the same descriptor.

A filesystem with no storage

A procfs mount contains no blocks and no inodes on disk. The kernel registers a set of operations for each file, and the file’s content is produced when it is read. The inode is created on lookup and discarded when the last reference goes away; there is no persistent object to which the file corresponds.

The generation model is pull, not push. Reading /proc/uptime causes the kernel to compute the value and format it into a buffer at that moment. Reading /proc/meminfo walks the memory counters and prints them. The file is a formatting function with a file-shaped interface, and the cost of a read is the cost of computing the value plus a copy to user space, not the cost of fetching a block from a cache.

This is why st_size is zero for most files in procfs. The size is not known before the content is generated, and for some files it is not knowable: the maps file for a process can grow as the process maps more memory. A program that calls stat, allocates a buffer of st_size, and reads once gets nothing useful. The correct approach is to read in a loop until read returns zero, growing the buffer as needed, which is the same loop required for pipes and terminals and for the same underlying reason: the file is a stream, and the size is a property of the stream’s end, not of its beginning.

Some files do report a size. /proc/kcore reports the size of the kernel’s address space, and /proc/config.gz on kernels built with the option reports the compressed size, because both are backed by something with a known extent. The inconsistency is itself the lesson: the metadata that ordinary programs rely on is present only when the backing object happens to have it.

Two families of file, with different lifetimes

The numeric directories under /proc correspond to processes, named by process ID. They appear when a process is created and disappear when it is reaped, so a path such as /proc/1234/status can fail with ENOENT between the lookup and the open. The entries inside a process directory are a mixture: status is text for humans and scripts, stat is a single line of space-separated fields whose format is fixed and documented, maps lists the memory mappings, fd is a directory of symbolic links to open files, cmdline is the argument vector with null bytes separating the arguments, and environ is the environment in the same encoding.

The other family is the global files at the top level: meminfo, cpuinfo, loadavg, uptime, version, mounts, sys as a symlink into sysfs. These are not tied to a process and persist for the life of the mount.

/proc/self is a symbolic link resolved per process: it points at the reading process’s own numeric directory. The resolution is not a fixed path but a property of the caller, which is the one place in the tree where the meaning of a name depends on who resolves it. /proc/thread-self does the same for the calling thread, and the thread directories are visible under the process directory as task/<tid>.

#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>

int main(void) {
    char buf[4096];
    int fd = open("/proc/self/stat", O_RDONLY);
    if (fd < 0) { perror("open"); return 1; }

    ssize_t total = 0, n;
    while ((n = read(fd, buf + total, sizeof buf - 1 - (size_t)total)) > 0) {
        total += n;
        if ((size_t)total >= sizeof buf - 1) break;  /* truncate deliberately */
    }
    buf[total] = '\0';
    printf("%s", buf);
    close(fd);
    return 0;
}

The stat line begins with the process ID and the command name in parentheses, and the command name can itself contain spaces and parentheses. Parsing it by splitting on whitespace is wrong for exactly that reason; the documented approach is to find the last ) in the line and parse the fields after it, because the name field is the only one that can contain arbitrary characters. This is a formatting decision that has caused a generation of parsers to be rewritten.

Where the file abstraction stops fitting

The first mismatch is seeking. A procfs file has no random access in general, because its content is generated in sequence. Seeking to the start of /proc/meminfo re-generates the content, and the values may differ from the previous read. Seeking into the middle of a generated file has no defined meaning beyond what the individual file’s implementation chooses to support, and many files simply do not implement it. A program that treats procfs as a regular file and uses mmap fails, because the mapping requires a page-backed object.

The second is that reads are not atomic with respect to the state they report. /proc/meminfo prints dozens of counters in one pass, and each is read as the formatter reaches it; the values do not correspond to a single instant. A program that computes free memory by subtracting two fields from one read can obtain a negative result, because the fields were sampled at different times. The stat file has the same property. There is no snapshot mechanism in the interface.

The third is that the same information is exposed in several formats with different guarantees. status is a key: value text file intended for readability and its field set has grown over time. stat is positional and its field count has also grown, with new fields appended so that existing positions stay valid, which is why parsers must count fields rather than assume a fixed number. statm reports memory in pages rather than bytes. A program that needs one number must choose a file and accept that file’s units and stability.

The fourth is that some files are not safe to read. /proc/kcore is the kernel’s memory as an ELF core file; reading it requires privilege and yields a view of kernel memory that changes underneath the reader. /proc/sysrq-trigger is write-only and its content is an action, not data. The uniform interface admits files whose read and write have side effects, which is a category that ordinary files do not have.

Writes, and the configuration surface

The sys subtree of procfs, usually mounted separately at /proc/sys, accepts writes that change kernel parameters. echo 1 > /proc/sys/net/ipv4/ip_forward sets a tunable; the write is parsed, validated, and applied, and the file’s subsequent read reflects the new value. The interface is the file write, but the semantics are a control operation, and the failure modes are those of a control operation: a partial write that leaves the parser with a prefix, a write of a value out of range that is rejected with EINVAL, a write that succeeds and takes effect only for new sockets.

The read-modify-write idiom that works for configuration files does not work here. Reading a sysctl, appending to the text, and writing it back is meaningless because the content is a single value rather than a document. Programs that manage these knobs must know the format of each one, and the formats are not uniform: some are integers, some are lists of integers, some are strings with restricted character sets.

This is the clearest case of the abstraction costing something. A dedicated ioctl or a typed configuration interface would express the value and its range. The file interface expresses only bytes, so every consumer must re-implement the parsing and every producer must document its grammar in prose. The benefit is that shell scripts and existing tools can read and write the knobs without a new library, and that benefit is why the design persists.

What the interface is actually good for

The strength of procfs is discoverability and composability with existing tools. grep, awk, sort and a shell loop can answer questions about a running system without any dedicated client, and a program can read the same files with the same open/read/close calls it uses everywhere else. The cost is that the answers are only as good as the file formats, and the formats are a compatibility surface that cannot be changed freely.

The other strength is that the information is in the namespace. A process’s descriptors appear as symbolic links under /proc/<pid>/fd, so the same path resolution and the same permission checks apply as for any other file. That is what lets a debugging tool list a target’s open files using readlink rather than a special system call, and it is why the descriptor model described in the first article becomes inspectable at all.

The limits are equally structural. procfs reports; it does not let a program wait for a change. There is no poll on /proc/meminfo that fires when memory pressure changes, and no way to subscribe to a process’s exit other than the signal and wait mechanisms. The files are a sampling interface, and a program that needs events must use the event interfaces. The same division appears in the process-group and session structures described in the job control article, where the kernel maintains state that is queried rather than pushed.