Signal Handling in Shell Scripts: Traps, Races, and Termination Forwarding

A shell script that traps SIGTERM and cleans up before exiting is a common pattern and a fragile one. The trap does not make the handler run when the signal arrives. It makes the handler run at the next point where the shell is able to run it, which is after the currently executing command has finished, and that delay is the source of most of the failures around script termination.

The handler runs between commands

The shell is a program that executes other programs. While a foreground child is running, the shell is blocked in wait, and a signal delivered to the shell interrupts that call. The shell then has a choice: run the handler immediately, or wait for the child to finish and run it afterwards. POSIX shells run it after the current command completes, which means a script blocked on a long-running child does not react to SIGTERM until the child exits on its own.

That is why the standard pattern for a supervising script uses wait explicitly:

#!/bin/sh
child=
term() {
    [ -n "$child" ] && kill -TERM "$child" 2>/dev/null
    wait "$child" 2>/dev/null
    exit 143
}
trap term TERM INT
sleep 3600 &
child=$!
wait "$child"

The wait builtin is interruptible: a signal delivered while it is waiting causes it to return immediately, and the handler then runs. The child, meanwhile, received nothing, because the signal went to the shell alone. Forwarding is the handler’s job, and the handler has to know which process to forward to.

The alternative is to give the child its own process group and signal the group. A script started as a job by an interactive shell already has a group of its own, so kill -TERM 0 from inside the script signals every process in that group, the script included. That is efficient and dangerous: if the script is not in its own group, as is the case for a script run non-interactively without job control, the group is the parent’s and the signal reaches processes the script does not own.

Signals ignored at startup cannot be trapped

POSIX imposes a rule that surprises anyone who tests a script only from an interactive shell: a signal that was ignored when a non-interactive shell started cannot be trapped or reset by that shell. The rule exists because the ignore disposition is the only mechanism a parent has for telling a child that this signal must not terminate it, and a child that could override it would break that contract.

The practical case is SIGINT under a job-control-less launcher. A process started with SIGINT ignored, which is what happens when a shell runs a background job, passes the ignore to the script, and the script’s trap on INT silently does nothing. The handler is installed in the shell’s table but the disposition is never changed. SIGQUIT has the same behaviour.

SIGKILL and SIGSTOP cannot be trapped at all, by anyone. A script cannot clean up after SIGKILL, which is why termination protocols distinguish a polite request from a forced kill and why a supervisor escalates from one to the other after a grace period.

The startup window

Between the kernel starting the interpreter and the trap command executing, the script has the default dispositions. A SIGTERM delivered in that window terminates it without running any cleanup. The window is small but it is real, and it cannot be closed from inside the script, because closing it would require the disposition to be set before the first line runs.

Two things reduce the exposure. The trap can be installed as the first executable statement, before any variable assignment that might block. And the script can arrange to do its real work in a single child, so that even an untrapped termination leaves nothing behind; if the only thing that survives is the child, and the child is in the same process group, a group-directed signal reaches it anyway. A wrapper that forks and then does bookkeeping is the shape that suffers most from this window, because the bookkeeping is what would have been skipped.

Exit status after a trapped signal

A handler that calls exit 143 reports the same numeric status as a process killed by SIGTERM, but it is not the same event. The parent sees a normal exit with status 143 rather than a signal death, and anything that inspects WIFSIGNALED, such as a supervisor, a test harness, or a shell reporting “Terminated”, sees the difference.

To reproduce the signal death, the handler resets the disposition and re-raises:

term() {
    trap - TERM
    kill -TERM $$
}

After trap - TERM, the default action is restored, and the self-directed signal terminates the process with the expected status. This matters when a wrapper sits between a supervisor and a program: the supervisor’s restart policy may distinguish a clean exit from a signal death, and a wrapper that converts every termination into exit 143 erases that distinction. The conventions around numeric statuses are covered in exit status semantics.

A handler can also run more than once. If two SIGTERM messages arrive before the handler completes, the shell may run the handler twice, and a cleanup routine that is not idempotent will fail the second time, typically by trying to remove a file that is already gone and exiting non-zero from inside the trap.

Traps that are not signals

Bash adds pseudo-signals: EXIT, ERR, DEBUG, and RETURN. They are not delivered by the kernel and cannot be sent from outside. EXIT runs when the shell exits for any reason, including a normal end of script, which makes it the natural place for cleanup. ERR runs after a command fails, subject to the same exemptions that limit set -e, described in the errexit discussion; the two mechanisms share the exemption list because ERR is defined in terms of it.

None of these are portable to a strict sh, and a script that uses them is depending on the shell that runs it. The EXIT trap is the one worth reaching for even in portable code, since POSIX defines trap ... 0 as running on exit.

When the script is the first process

A shell running as process id 1 in a container has different signal semantics from an ordinary process. Linux does not apply default signal actions to the init process: a signal whose disposition is default is discarded rather than acted on. A shell that has not installed a handler for SIGTERM therefore ignores it entirely, and the container takes the full grace period before being killed.

This is the mechanism behind a common operational puzzle: a container whose entrypoint is a shell script does not stop promptly. The remedy is either to install handlers for the signals that matter, or to exec the real program so that it replaces the shell and receives signals directly. The exec form is preferable when no cleanup is needed, because it removes a process from the tree and eliminates the forwarding problem rather than solving it. It relies on the fact that exec preserves the pid, as described in the execve contract.

The limits of the mechanism

Traps give a script a way to observe termination, not a way to make termination synchronous. A handler cannot interrupt a blocked system call in a child, cannot run while the shell is executing a builtin, and cannot be relied on at all for the two signals that cannot be caught. The window before the first trap remains, and the rule about inherited ignores removes the option of trapping SIGINT in some launch contexts entirely.

What remains is a mechanism that works well for the case it was designed for: a script that spawns a known set of children and wants to forward a termination request to them, wait for them, and then exit with a status that reflects what happened. Anything beyond that, including ordering guarantees, cleanup under SIGKILL, and reliable handling of signals that arrived before the script started, is outside what the shell interface can provide.