POSIX Mode Bits, umask, and the ACL Mask Interaction

A Unix file carries a twelve-bit permission word in its inode. Nine of those bits are the familiar owner, group, and other triples; three are the setuid, setgid, and sticky flags. Almost everything surprising about Unix permissions follows from the fact that this word describes three classes of principal, while the principals who actually need access are usually four or more. The ACL extension was added to cover the gap, and the way it was grafted onto the existing bits is the source of the mask confusion that still costs operators hours.

Nine bits and three classes

The permission word is conventionally written in octal. The low three bits are other: read 4, write 2, execute 1. The next three are group, and the next three owner, so a file readable by everyone and writable only by its owner is 0644. The top bits are 04000 for setuid, 02000 for setgid, and 01000 for the sticky flag.

Read on a regular file permits reading bytes. Write permits modifying and truncating it, and notably does not by itself permit unlinking it. Execute permits the kernel to load it as a program image; on a directory, execute means the right to traverse the directory as a path component, while read means the right to list its entries. Those two directory rights are independent, which is why a directory with mode 0711 lets a caller open a known file inside it but not discover the names of the files it does not already know.

Group membership is decided by the process credentials: the effective group and the supplementary group list. A process whose group list contains the file’s owning group is in the group class. The owner class wins over the group class, and the group class wins over other. A file owned by a user who is also in the owning group is evaluated as owner, so tightening the group bits never restricts the owner.

The three-class model is coarse in one specific way: it cannot grant different rights to two different people who are not the owner and share no group. Creating a group per pair of collaborators is the traditional workaround, and it consumes a namespace and requires privilege to administer.

What the ACL extension adds, and where the mask comes from

POSIX ACLs, standardized in the withdrawn POSIX.1e draft and implemented broadly anyway, attach a list of entries to the inode, usually stored in the extended attribute system.posix_acl_access. Each entry names a tag and a permission triple. The tags are ACL_USER_OBJ, ACL_USER with a named user, ACL_GROUP_OBJ, ACL_GROUP with a named group, ACL_MASK, and ACL_OTHER. An ACL with only the three object entries and no mask is exactly equivalent to the mode bits, which is what makes the extension backward compatible.

The mask entry is the awkward part. When an ACL contains any named user or named group entry, the mask becomes mandatory, and it caps the permissions granted by every named user entry, every named group entry, and the owning group entry. It does not cap the owner entry or the other entry. A named user entry of rwx under a mask of r-x grants read and execute only.

The mask exists because the nine mode bits have nowhere else to put the group class. On a file with an ACL, the group bits of the mode word display and control the mask rather than ACL_GROUP_OBJ. So chmod g-w on such a file does not revoke write from the owning group specifically; it clears write from the mask, which revokes it from the owning group and from every named user and named group entry at once. chmod 640 rewrites the mask to r--, silently narrowing entries the caller never intended to touch. This is the single most common way an ACL configuration is destroyed by a routine permission change.

# The mask is the ceiling for named entries; set it explicitly.
setfacl -m u:alice:rw-,g:analysts:r--,m::rw- report.csv
# chmod now edits the mask, not the owning-group entry:
chmod g-w report.csv          # mask becomes r--; alice loses write

Tools that understand ACLs present the effective permission separately from the entry, precisely because the two differ whenever the mask binds. Reading a listing that shows only the mode word tells the operator nothing about a named entry that the mask has clipped.

umask is a filter on creation, never a restriction afterwards

The umask is a per-process bitmask applied at file creation. The kernel computes the new file’s mode as the requested mode with the umask bits cleared, so a process with umask 022 that asks for 0666 receives 0644. The filter applies to open with O_CREAT, to mkdir, to mknod, and to the creation of POSIX message queues and shared memory objects; it does not apply to chmod, and it does not affect files created by another process.

Two properties of the umask cause trouble. It is inherited across fork and preserved across exec, so a service that sets a permissive umask for one operation leaks that setting into every child it later spawns. And it only clears bits: a umask cannot add permissions the caller did not request, and a mode argument of 0600 stays 0600 under any umask.

The default ACL on a directory changes the arithmetic. When a directory carries a default ACL, files created inside it take their access ACL from the default entries, and the mode argument and umask are applied as a restriction on that inherited ACL rather than as the primary source. The umask still clears bits from the inherited entries in the implementations that follow the POSIX.1e draft, which means a permissive default ACL combined with a restrictive umask produces something neither setting alone would predict.

Inheritance, ownership, and the setgid directory

A new file’s owner is the creating process’s effective user, and its group is the process’s effective group, unless the containing directory has the setgid bit set. On such a directory the new file inherits the directory’s group, and subdirectories inherit both the bit and the group, so a shared tree can keep a consistent group without each creator changing groups first. The setgid bit on a regular file is a different mechanism entirely: it requests that the process run with the file’s group when executed.

Ownership changes have their own rules. Only a privileged process can give a file away to another user; an unprivileged owner may change the group only to a group they belong to. Both restrictions exist because ownership transfer is otherwise a way to plant a setuid binary that a more privileged user will later execute.

The sticky bit on a directory restricts removal and renaming of entries to the file’s owner, the directory’s owner, or a privileged process. It does not restrict creation. A directory with mode 1777 is writable by everyone for creating files, while deletion of another user’s file is refused. Without the sticky bit, write permission on the directory implies the ability to delete any entry in it, since unlinking is an operation on the directory.

Failure modes that survive in ordinary use

The sticky bit does not protect against a determined attacker replacing a file they own in a shared directory with a symbolic link that a privileged program will later follow; that is a check-then-use problem rather than a permission-bit problem, taken up in the reference monitor discussion.

Root bypasses read, write, and execute checks, with one exception: executing a file still requires at least one execute bit set somewhere in the mode word. A root-owned script with mode 0644 is not executable even by root, which surprises anyone who expects privilege to imply every right.

Network filesystems diverge. NFSv4 ACLs are a different model with inheritance flags, deny entries, and an explicit ordering of entries, mapped onto POSIX ACLs with loss. A file created locally and exported can present different effective permissions to the same user depending on whether the client or the server evaluates the check, and whether the client’s group list is trusted.

Finally, ACLs and mode bits are metadata, not data. They are enforced by the kernel for local access, and they are advisory for anything that reaches the storage by another route: a backup agent reading the raw device, a container runtime mounting the host path, an object store that maps its own policy language onto the exported view. The authorization question is always answered relative to a specific enforcement point, and the file mode is only one such point.

The scaling limit is worth stating because it drives most migrations away from this model. An access control list on a file holds a bounded number of entries, typically a few dozen before the storage format’s limits become relevant, and each entry is a specific principal. A directory serving ten thousand users with per-user distinctions cannot be expressed, so the practical workaround is one group per distinction, and the group list becomes the large object. The kernel compares a process’s group list against each file’s entries, so a user in a thousand groups costs a scan that grows with the interplay of both counts. Systems with many principals and many objects per directory move to a model that evaluates rules rather than matching lists, which is where attribute-based evaluation enters and why the file-mode model remains the right choice only for the local, small-scale case it was designed for.