Distributed Systems · Time and Ordering
Monotonic and Civil Clocks in Running Programs
Why interval measurement needs a clock that never goes backwards, how leap seconds and suspend break civil time, and which timeout a kernel or runtime actually uses.
A general-purpose operating system exposes at least two clocks to user space, and they answer different questions. One says what time it is. The other says how much time has passed. Confusing them is a common source of wrong behaviour in code that measures, schedules, or times out, and the failure is often invisible until a clock correction or a leap second exposes it.
Two clocks with different contracts
On POSIX systems the interface is clock_gettime, which takes a clock identifier and fills a struct timespec with seconds and nanoseconds. The two identifiers that matter here are CLOCK_REALTIME and CLOCK_MONOTONIC.
CLOCK_REALTIME is civil time: the number a person would recognize, aligned to UTC, subject to adjustment by the time synchronization daemon. It can move forward and, when a correction or a leap second requires it, backwards — the step and slew adjustments described in The NTP Four-Timestamp Exchange and Its Asymmetry Limit are what move it. Its resolution is typically one nanosecond in the reported value, but the underlying hardware resolution is coarser, and the value is a cached reading updated from a counter.
CLOCK_MONOTONIC is elapsed time since an unspecified starting point, usually boot. It never decreases. It is not affected by manual clock changes or by network time corrections; the kernel maintains it from a hardware counter and a rate correction, and a step to civil time does not propagate into it. Its absolute value is meaningless to a human, since the epoch is arbitrary and not required to correspond to any fixed date.
The contract difference is exactly the one that matters. A duration is the difference of two readings of the same clock, and that difference is only meaningful if the clock is monotone and runs at a rate close to real time. Civil time satisfies neither guarantee under adjustment; monotonic time satisfies both, and sacrifices the ability to say what the date is.
Linux adds a third variant, CLOCK_MONOTONIC_RAW, which reports the uncorrected hardware counter without the frequency adjustment applied to CLOCK_MONOTONIC. It is useful when the consumer wants to apply its own frequency estimate, and it drifts relative to civil time because the raw crystal does.
What a leap second does to a civil clock
UTC is not uniform. It is kept close to the Earth’s rotation by inserting or deleting a second when the difference between atomic time and astronomical time approaches a threshold. The insertion is announced in advance, and it happens at the end of a UTC day, so the final minute of that day contains 61 seconds instead of 60.
A naive implementation of civil time that counts seconds since an epoch cannot represent the extra second. The common approach is to repeat a second: the value 23:59:59 appears twice, and any program that compares timestamps around that boundary sees two distinct instants with the same civil value. Sorting is then ambiguous for one second, and a duration computed across the boundary is off by one second.
The alternatives in use are to smear the extra second over a window, running the clock slightly slow for some hours before or after the event, which keeps the clock monotone and the value unique at the cost of a small offset during the window; or to stop the clock for a second, which keeps values unique but breaks monotonicity in the sense of rate. Both are policy decisions that a host makes, and the choice affects every program on the host, since they all read the same civil clock.
The effect on a timeout is concrete. A program that arms a timer for 30 seconds against civil time and crosses a backwards correction of 1 second may wait 31 seconds of real time, because the clock it is measuring against lost a second. A program that arms the same timer against monotonic time waits 30 seconds regardless of what happened to the date.
Suspend, virtualization, and the meaning of elapsed
A monotonic clock counts while the machine is running, and the definition of “running” is where implementations differ. On a laptop that suspends for eight hours, CLOCK_MONOTONIC on Linux does not advance during suspend, because the clock is derived from a counter that stops. CLOCK_BOOTTIME does advance, because it includes time spent suspended. Neither is wrong; they answer different questions, and a program that wants “time since I started this operation, including any suspension” must ask for the second.
Virtual machines add another layer. A guest reading a paravirtualized clock may see time jump when the host pauses the guest for scheduling, migration, or snapshot. A guest that was suspended and resumed can find that its monotonic clock is behind the host’s, and a guest that was migrated can find it ahead. Hypervisors usually offer a clock that accounts for this, but the guest must opt into it and must know whether it wants elapsed wall time or elapsed execution time.
#include <time.h>
/* Duration between two monotonic readings, in seconds. */
double elapsed(struct timespec a, struct timespec b) {
return (double)(b.tv_sec - a.tv_sec)
+ (double)(b.tv_nsec - a.tv_nsec) / 1e9;
}
/* Correct usage: measure with the monotone clock. */
struct timespec t0, t1;
clock_gettime(CLOCK_MONOTONIC, &t0);
do_work();
clock_gettime(CLOCK_MONOTONIC, &t1);
double seconds = elapsed(t0, t1);
The subtraction is safe because the second reading is never less than the first, but the conversion to double loses precision once the magnitudes are large; on a host up since a recent boot the difference is small and the subtraction of two large integers is exact, so the loss is only in the final division. Code that needs nanosecond exactness should keep the difference in integer nanoseconds.
Timeouts in kernels and runtimes
The choice of clock is visible in the interfaces that schedule work. A kernel timer is armed against a monotonic clock in essentially all modern implementations, because a timer that expires early or late when the wall clock is corrected is a correctness bug in the scheduler. The Linux timer wheel and the high-resolution timer subsystem both work in monotonic time, and a timerfd is created with an explicit clock — CLOCK_MONOTONIC for a deadline measured from now, CLOCK_REALTIME for an alarm that must fire at a civil time — so the choice is visible at the call site rather than buried in the implementation.
The same distinction appears in the standard library of most languages. A deadline is usually represented as an instant on a monotonic timeline, and converting it to a civil time is a separate, lossy operation. Runtimes that expose only a civil clock for timeouts inherit the correction problem, which is why a language runtime’s timer facility is often documented in terms of a monotonic source even when the API looks like a date.
The distinction also governs cache expiry and lease renewal. A lease with a deadline expressed in civil time expires late if the clock jumps backwards, which can extend a lease beyond the holder’s intent; a lease expressed in monotonic time expires after a fixed elapsed interval regardless of corrections. Distributed leases have the additional problem that the two sides measure elapsed time on different machines, which is why a lease protocol must bound the clock rate difference rather than assume a shared clock, as discussed in Bounded Clock Uncertainty and Commit-Wait.
Formatting and storage
Civil time remains the right choice for anything a person reads or a policy refers to. A log record’s timestamp, a retention window, a certificate’s validity period, and a scheduled job’s time of day are all civil quantities, and converting them to monotonic time would lose the meaning. The conversion between civil time and a local calendar date goes through a time zone database rather than through the kernel: the TZif format defined in RFC 8536 stores the transition times and offsets for each zone, and a program that renders a timestamp as local time is reading that data, not the clock. A monotonic reading cannot be converted to a date, because its epoch is unspecified, but a civil instant can be compared with another civil instant as long as both are read from clocks with known error bounds.
The storage rule that follows is to record which clock produced a value. A timestamp without a clock identity is ambiguous, and the ambiguity matters when a value is compared against one produced elsewhere. A duration is safe to store as a difference of monotonic readings, but it is not safe to compare with a duration measured on another host without knowing that host’s rate error.
The rule that covers most cases is short: measure intervals with a monotonic clock, and record events with a civil clock. The first is immune to correction; the second is meaningful to humans. Mixing them produces the failures described above, and the failures appear at exactly the moments when the system is under the most stress.
References
- clock_gettime(2) — Linux manual page — Linux man-pages project
- The Open Group Base Specifications Issue 7 — clock_gettime — IEEE Std 1003.1 (POSIX)
- RFC 8536: The Time Zone Information Format (TZif) — IETF