Observability · Tracing
Span Anatomy: Trace IDs, Span IDs, and Parent Links
How a trace is assembled from spans, why two identifiers with different scopes are needed, and what a span record must carry for the tree to be reconstructible offline.
A trace is not a stream of events that some central component stitches together as they arrive. It is a set of independent records, each written by the process that executed a unit of work, and the tree shape emerges only when those records are read back and joined by identifier. That design choice, made because no collector can be in the request path of every service, dictates almost everything about the span format: identifiers must be globally unique without coordination, parents must be named explicitly because arrival order says nothing, and every record must be self-describing enough to be interpreted by a reader that never saw the writer.
Two identifiers with different scopes
A span carries a trace identifier and a span identifier. They look similar, both are opaque fixed-width values, and the distinction between them is the first thing to get right.
The trace identifier is shared by every span belonging to one logical operation, from the first ingress point to the last database call. Its width is 16 bytes, rendered as 32 lowercase hexadecimal characters. The width is a compromise: 8 bytes would collide at a rate that becomes visible once an organisation accumulates billions of traces per day, and 32 bytes wastes space in every record and every header for no additional safety. Sixteen bytes gives 2^128 values, and the specifications require the value to be non-zero, which leaves 2^128 - 1 usable. Because the value must be unique without any coordination between the processes that mint it, it is generated from a random source rather than a counter. The birthday bound is the relevant arithmetic: collision probability stays negligible until the number of traces approaches the square root of the space, which is 2^64.
The span identifier scopes one unit of work within that trace. It is 8 bytes, 16 hexadecimal characters, also non-zero. Eight bytes are sufficient because the uniqueness requirement is local: a span identifier only has to be distinct among the spans of its own trace, and a single trace rarely exceeds a few million spans. The pair of trace identifier and span identifier is the primary key of the record.
Neither identifier encodes structure. There is no timestamp, no host, no sequence number, no service name inside them, and nothing can be parsed out of the hex. This is deliberate. Any encoding of metadata into the identifier would make the value predictable, and predictability invites collisions when two processes generate identifiers in the same clock tick or from the same seeded generator. Keeping the values opaque also lets an implementation switch from random generation to a time-ordered scheme, such as a 64-bit timestamp prefix followed by random bits, without breaking any consumer. The time-ordered variant preserves uniqueness while making identifiers sort roughly by creation time, which helps storage engines that cluster by key, but the sort order is an implementation detail rather than a contract.
Parent links make the tree, not the transport
Each span names its parent by span identifier, and the parent is always in the same trace. That single field is what turns a bag of records into a tree. A span with no parent is a root, and a trace may legitimately have several roots if a request enters the system through more than one path or if an asynchronous job is attached to a trace without being caused by a specific span.
The parent pointer is carried in the request, not derived on the receiving side. When a client calls a server, the client’s outgoing span becomes the parent of the server’s incoming span, and the server learns the parent’s identifier from the request header rather than from any server-side state. This is why tracing survives process restarts, load balancer hops, and queue decoupling: the relationship is data, transported alongside the work. The wire format for that transport is specified separately, and the header layout has its own constraints; the traceparent header format covers the exact byte fields, and the practical difficulties of moving that header through proxies, message brokers, and thread pools are the subject of propagating context across process boundaries.
Not every causal relationship is a parent-child relationship. A batch consumer that processes two hundred messages from a queue in one poll has two hundred causal predecessors and one span. Duplicating the consumer span per message inflates the trace; picking one parent and discarding the rest loses information. The span model answers with links: additional references to other spans that record causal association without affecting the tree shape. A link carries its own trace identifier and span identifier, so it can cross trace boundaries entirely. This is the mechanism used for fan-in, for retries that produce a new trace, and for any join where the many-to-one relationship would otherwise be misrepresented.
The distinction matters for query. Tree traversal follows parent pointers and stays inside one trace. Link traversal is a graph walk that can leave the trace, and most storage schemas index links separately, if at all.
What a span record carries beyond its identifiers
Identifiers alone reconstruct shape but not meaning. A minimal useful record holds a name, a start timestamp, an end timestamp, and a status. The name is low-cardinality by design: an operation such as GET /orders/{id} or SELECT orders, not a formatted string with the identifier substituted in, because the name is what aggregation groups by. The status is a small enumeration, and the common model distinguishes an unset status from an explicit error, since a span that never set a status is not evidence of success.
Attributes are key-value pairs attached to the span. Their cardinality is the recurring design problem: a value like an HTTP status code or a peer service name has a bounded set of values and aggregates cleanly, while a value like a user identifier or a raw SQL statement has unbounded cardinality and turns every aggregation into a unique key. The conventions that name attributes exist precisely to keep the bounded ones consistent across services, so that a query for a status code does not have to try three spellings.
A span also carries a kind, which tells the reader what the span’s position in the call means. The common set is internal, server, client, producer, and consumer. The kind is not derivable from the tree, because a client span and a server span sit at the same depth from opposite sides of the same network hop, and a reader that wants to compute network latency by subtracting one from the other needs to know which is which. Kinds also drive correctness checks: a well-formed trace alternates client and server spans across each hop, and a run of two server spans in a row usually means a missing propagation point.
Timestamps deserve a note because they are the field most likely to be wrong. The start and end are recorded by the local clock of the writing process, and no protocol synchronises those clocks. The duration is computed as a difference of two readings from the same clock, so it is usually sound even when the absolute time is off by milliseconds. Absolute comparison across spans is not sound in the same way, which is why clock skew and the arithmetic of span durations treats the two uses separately.
Events and the cost of self-description
A span may carry events: timestamped annotations inside its interval, each with a name and optional attributes. An exception recorded on a span is usually an event with a well-known name and attributes holding the type and message. Events exist because not every interesting moment is worth a child span. Opening a connection, retrying, and emitting a log line are moments, not operations, and giving each one a span identifier would multiply the record count without adding a measurable interval.
The cost of self-description is that a span record is large relative to the work it describes. A record with identifiers, timestamps, kind, status, a dozen attributes, and a few events runs to several hundred bytes before compression, and a service handling thousands of requests per second produces spans faster than most networks and disks can absorb them at full fidelity. That is the reason sampling exists at all, and the reason the decision of which traces to keep is a design problem rather than a configuration afterthought; head-based sampling and tail-based sampling are two different answers to it.
What the model cannot express
The tree assumes each span has one interval and one parent. Work that is genuinely concurrent inside one span, such as a server handling several requests on a shared event loop, appears as overlapping sibling spans whose individual durations do not sum to the parent’s. Work that spans a process boundary without a request, such as a background compaction triggered by a threshold rather than by a caller, has no natural parent and becomes a root. Long-lived spans, such as a subscription that stays open for hours, distort any duration-based aggregation and usually need to be broken into bounded pieces.
None of these are defects to be fixed by extending the format. They are places where the underlying question, “what caused this and how long did it take”, does not fit a strict tree, and the model’s answer is to record the awkward case explicitly through links and roots rather than to pretend the tree is complete.
References
- Trace Context — W3C Recommendation
- Dapper, a Large-Scale Distributed Systems Tracing Infrastructure — Sigelman et al., Google Technical Report 2010
- OpenTelemetry Trace API specification — OpenTelemetry project