Searcher Snapshots and Lock-Free Concurrent Readers

A search index changes constantly: documents are added, segments are merged, deletes are applied. Yet queries must never observe a torn state, a segment set that is half-old and half-new. The mechanism that guarantees this is the searcher snapshot: an immutable, reference-counted view of a specific segment set that each query acquires at start and releases at end, and that the writer swaps atomically when it publishes a change. Everything about concurrent search rests on that single idea.

The searcher as an immutable view

A searcher is not the index itself; it is a view over the index. It holds the list of segment files that were visible when it was created, along with the loaded term dictionary and any per-segment metadata such as delete bitvectors. Because a searcher never changes after creation, a query that holds one can walk postings lists, read dictionaries, and consult bitvectors without any locking: the data it reads cannot be mutated out from under it.

The writer, meanwhile, produces a new searcher for every refresh or commit. The new searcher points at the new segment set. The publish step is a single atomic pointer swap: the current searcher reference is replaced with the new one, using a volatile reference or an atomic compare-and-swap. A query that reads the pointer once, at the start of its execution, holds that searcher for its entire lifetime, so the swap happening mid-query does not affect it.

Why files survive the swap

The swap changes which files are referenced, not which files exist. A segment file that is dropped from the new searcher, because its contents were merged into a larger segment, must stay on disk until every in-flight query that still references it has finished. This is reference counting over files: each searcher holds a reference to each of its segments, the count is incremented when a searcher is created and decremented when it is closed, and a segment file is deleted only when its count reaches zero.

The failure mode this prevents is a query that faults in a page from a file the writer has just deleted. On POSIX systems, an open file descriptor keeps a deleted file’s data alive until the descriptor closes, so a query that has opened its segment files survives their unlink; the reference count is what decides when the unlink happens. Getting the count wrong, by closing a searcher that is still in use or by leaking one that is finished, produces either a deleted file read by an active query or a permanently stranded file that is never reclaimed.

The counting is itself a subtle concurrency problem. A searcher is handed out to many threads, and its reference count is decremented by the last thread to finish using it. If two threads decrement the count at once without synchronization, the count can underflow and the searcher, and its files, are freed while still in use. The standard technique is an atomic decrement where the thread that observes the count reach zero is the one responsible for the teardown, a pattern that avoids the double-free without holding a lock on the read path. The single atomic instruction is the only synchronization a query thread ever performs, and it is the price of the lock-free reads the immutable design otherwise grants.

The cost of many snapshots

Each refresh creates a new searcher, and each searcher holds references to every segment. The cost is not the pointer swap, which is a single atomic write, but the teardown: the old searcher cannot be fully closed until the last query holding it finishes, so under continuous traffic the system carries several generations of searchers at once, each pinning a slightly different segment set. The pinned segments keep deleted and merged-away files alive, which delays the disk space reclamation that the merge operation was supposed to provide.

This is the concrete price of zero-downtime updates: the space is reclaimed late, on the schedule of the slowest in-flight query, not on the schedule of the merge. A query that runs for seconds, whether because it is broad or because it is waiting on cold disk, pins the old segment set for that entire duration and defers the deletion of every file the merge removed.

The generations also consume open file descriptors. A searcher holds a descriptor for each of its segment files, and a system that has many generations of searchers in flight multiplies its descriptor count by the number of generations. On a system with a fixed descriptor limit, this is a real ceiling: too many un-closed searchers exhaust the limit and new file opens fail, which surfaces as a mysterious failure long after the merges that caused it. The mitigation is to bound the number of live searcher generations and to reuse a searcher where a fresh one is not needed.

Warming the new snapshot

A freshly published searcher is cold: its files have not been read, its dictionary pages are not in the page cache, and its hottest postings lists are not resident. The first queries to hit the new snapshot therefore pay the full cost of faulting everything in, which shows up as a latency spike right after each refresh. Systems mitigate this by warming, a best-effort pass that pre-reads the new searcher’s dictionary and the most frequently queried terms before, or immediately after, the swap, so that the first real query finds its data already in memory.

Warming is a trade against the refresh interval. Warming a large index on every one-second refresh would consume more I/O than the refresh itself saves, so warming is usually throttled or limited to a small set of hot terms, accepting that the long tail of the new snapshot stays cold until some query touches it.

The warm set is itself a data structure: a list of the most frequently queried terms, maintained from query logs and capped at some size, read at warm time and walked to pre-touch the corresponding postings lists. Because the warm set is a heuristic snapshot of past queries, it can miss the term that the very next query asks for, and warming then does nothing for that query. The guarantee warming provides is therefore not “the snapshot is warm” but “the snapshot is warm for the queries that historically mattered,” which is a weaker but still valuable property: it removes the steady-state cold-start cost and leaves only the long-tail misses.

Snapshot isolation as a consistency model

The snapshot mechanism gives queries a specific guarantee: each query sees a consistent point-in-time view of the index, and the view is stable for the query’s duration. This is read-your-writes within a single searcher, but it is not linearizability across refreshes. A query that starts just before a refresh sees the old data, even if the refresh completed while it ran; a query that starts just after sees the new data. There is no ordering guarantee between a refresh and a concurrent query, which is exactly why caches keyed to segment generations are valid: they are validated against the same snapshot boundary that queries observe.

The searcher snapshot is the quiet workhorse that makes everything else in this cluster look simple. Immutable segments make the snapshot cheap, the atomic swap makes publication instant, and reference counting makes the teardown correct; together they let a continuously changing index serve queries without a single lock in the read path.