The Terminal Line Discipline: Canonical Mode, Echo, and Raw Input

A program that reads from a terminal does not read keystrokes. It reads whatever the kernel’s line discipline has decided to hand it, after that layer has echoed the characters back, applied editing commands, translated line endings, and possibly converted some of them into signals. The layer is not part of the terminal emulator and not part of the application; it lives in the kernel, and its configuration is what distinguishes a shell prompt from a full-screen editor.

The layer between the device and read

The tty subsystem has three parts: the device driver that talks to the hardware or to a pseudo-terminal master, a line discipline that processes the byte stream, and the interface the process uses, which is a pair of read and write operations on a file descriptor. The line discipline is configured through a termios structure containing input flags, output flags, control flags, local flags, and an array of control characters.

tcgetattr reads the current settings and tcsetattr writes them. The settings belong to the terminal device, not to the process, so two programs reading the same terminal share one configuration; the second one to change it affects the first. That shared ownership is why a program that crashes without restoring the saved settings leaves the terminal in whatever state it last set, and why stty sane exists as a recovery command.

The distinction between the terminal and the file descriptor matters. A process can have its standard input redirected from a file while still having a controlling terminal, which is what /dev/tty refers to. A program that needs to talk to the user regardless of redirection opens /dev/tty rather than reading descriptor 0.

Canonical mode is a line editor in the kernel

By default the terminal is in canonical mode, indicated by ICANON. In this mode the driver accumulates input into a line buffer and delivers it to the reading process only when a line delimiter arrives. Until then, the process sees nothing at all: a read on a canonical terminal blocks until a complete line is available, and then returns the whole line including its terminating newline.

While the line is being accumulated, the driver performs editing. The erase character, usually backspace or delete, removes the previous character from the buffer. The kill character removes the entire line. Word-erase removes the previous word. None of these are seen by the application; they are processed and discarded, and the application receives the edited result.

The line buffer has a bounded size, and the bound is small. POSIX requires that a canonical input line be able to hold at least 255 bytes, and exceeding the limit produces a bell or a dropped character depending on the implementation. A paste longer than the limit is truncated, which is why pasting a long command into a terminal can silently lose its middle.

One character deserves separate mention. The end-of-file character, Ctrl-D by default, does not insert anything and does not close the descriptor. In canonical mode it causes the driver to make whatever is currently in the buffer available to the reader immediately, without a newline. Pressing it on an empty line therefore produces a zero-length read, which the reading program interprets as end of file. That is the whole mechanism behind Ctrl-D ending a shell session, and it explains why Ctrl-D in the middle of a line does nothing visible.

Characters that never reach the program

Several characters are consumed by the driver before any application sees them. The interrupt character, VINTR, generates SIGINT for the foreground process group; the quit character generates SIGQUIT, which by default also produces a core dump; the suspend character generates SIGTSTP. All three are enabled by the ISIG local flag.

This is why a program cannot read Ctrl-C as a byte. The driver has already converted it into a signal, and the byte itself is not placed in the input stream. A program that wants to see it must clear ISIG, at which point Ctrl-C becomes an ordinary byte with the value 3 and no signal is generated. That is precisely what a full-screen editor does so that it can implement its own interrupt handling. The signal target is the foreground process group, not the individual process, which is why one keystroke reaches every stage of a pipeline.

Clearing ISIG means the program must implement interrupt handling itself, since the driver no longer converts the interrupt character into a signal. A full-screen editor that clears it and then fails to handle the byte leaves the user with no way to stop the program from the keyboard.

Flow control adds two more invisible characters. With IXON set, Ctrl-S stops output and Ctrl-Q resumes it, and neither is delivered to the reader. A program that expects to receive every byte the user types will lose these two, along with the editing characters and the signal characters, unless it disables the corresponding flags.

Input translation is a third source of invisible changes. ICRNL converts a carriage return to a newline on input, which is what makes the Enter key work on terminals that send CR. ONLCR converts a newline to carriage-return plus newline on output, which is what makes text display correctly on terminals that need both. ISTRIP clears the eighth bit of every input byte. Any of these will corrupt a binary protocol carried over a terminal, and all of them are on by default.

Raw mode and what it switches off

Raw mode is not a single flag; it is a combination. The usual implementation clears ICANON, ECHO, ISIG, and IEXTEN from the local flags, clears IXON, ICRNL, and the other input translations, and clears output post-processing. The C library provides cfmakeraw to perform this set of changes, and each of them has to be considered separately because each has a different consequence.

#include <termios.h>
#include <unistd.h>

struct termios saved, raw;
tcgetattr(STDIN_FILENO, &saved);
raw = saved;
cfmakeraw(&raw);                 /* clears ICANON, ECHO, ISIG, IEXTEN, IXON, ... */
raw.c_cc[VMIN]  = 1;             /* return as soon as one byte is available */
raw.c_cc[VTIME] = 0;             /* and never time out */
tcsetattr(STDIN_FILENO, TCSAFLUSH, &raw);
/* ... read keystrokes one at a time ... */
tcsetattr(STDIN_FILENO, TCSAFLUSH, &saved);   /* restore before exiting */

Clearing ICANON changes the read semantics from “one line” to “whatever is available”, bounded by two control values. VMIN sets the minimum number of bytes a read returns; VTIME sets a timeout in tenths of a second. With VMIN at 1 and VTIME at 0, a read blocks until at least one byte arrives. With VMIN at 0 and VTIME at 5, a read returns after half a second even if nothing arrived, which is how a program implements an idle timeout without a separate timer. The four combinations of zero and non-zero values are all meaningful, and the ordering rule is that the timer starts when the read begins and is reset by each byte received when VMIN is non-zero.

Clearing ECHO means the program must echo characters itself if the user is to see them. A shell in raw mode that forgets to do so appears to accept no input at all, which is a common symptom of a program that entered raw mode and then failed.

Restoring state and the cost of not restoring

The saved termios structure should be restored on every exit path, including the ones taken after a signal. The usual implementation saves the structure before changing anything, installs handlers for the terminating signals, and calls tcsetattr with TCSAFLUSH in the handler and again on normal exit. TCSAFLUSH discards input that has been received but not read, which prevents a half-typed line from being interpreted by the shell after the program exits; TCSANOW applies the change immediately without discarding.

A program killed by SIGKILL cannot restore anything. The terminal is left in raw mode, echo is off, and the shell that regains control cannot display what is typed. The recovery is to run stty sane or reset blind, which is possible because the shell is still reading from the same descriptor even though the display is unusable.

The cost of raw mode is a system call per keystroke in the worst case. Canonical mode amortises that cost by delivering whole lines, so a program that reads interactively in canonical mode performs one read per line while a raw-mode program performs one per character. For a text editor that is the correct trade: latency per keystroke matters more than syscall count.

Why a terminal is not a pipe

The line discipline is the reason a program can behave correctly when reading from a file and incorrectly when reading from a terminal, or the reverse. A pipe delivers bytes as they are written and never edits them; a terminal delivers lines, echoes, translates, and generates signals. The two are both file descriptors and both satisfy read, which is exactly why the difference is easy to miss.

A program that needs to know which it has can ask with isatty. The answer changes what the sensible default behaviour is: prompting is appropriate on a terminal and inappropriate when input is a file, and the choice of buffering mode is made on the same basis. Detecting the terminal and adapting to it is the difference between a tool that composes in a pipeline and one that has to be worked around.