Information Retrieval · Cluster
Search Operations
Index updates and query latency.
- 01 The Document Add Path into an Inverted Index How an incoming document is tokenized, assigned a document id, and appended to the postings lists it matches, and what the ordering invariants on those lists cost.
- 02 Segment-Based Indexing and the Merge Operation Why search indexes are built as many small immutable segments rather than one mutable structure, and how the merge step reclaims dead postings and bounds segment count.
- 03 Deletion via Tombstones and the Delete Bitvector Why a search index does not remove a document immediately, how a tombstone marks a document dead, and what a query does when it encounters one.
- 04 Near-Real-Time Refresh and the Commit Boundary How an index makes newly added documents searchable within seconds, and the distinction between committing to the filesystem cache and committing to durable storage.
- 05 Query-Time Cost and the Conjunctive Fast Path What dominates the latency of a term-based query, how a conjunction of rare and common terms is evaluated, and where the unavoidable costs of scoring live.
- 06 Query and Filter Caches in a Search Engine Which parts of a query can be cached without corrupting results, how a filter cache keys on the filter itself rather than the query, and why invalidation is the hard part.
- 07 Merge Scheduling and Latency-Bounded Throttling How a search engine decides which segments to merge and when, and how it keeps merge I/O from spiking query latency while still reclaiming space fast enough.
- 08 Searcher Snapshots and Lock-Free Concurrent Readers How a search engine publishes a new index version to running queries without pausing them, using copy-on-write searcher snapshots and reference counting.
- 09 Write-Ahead Logging and Crash Recovery in Search How a search index survives a crash without losing recently added documents, by logging mutations to a sequential write-ahead log and replaying it after restart.
- 10 Shard Routing and Replica Consistency in Distributed Search How a search index is split across many machines, how a query fans out to shards and merges their results, and how replicas stay consistent with their primary shards.