Inference Systems · Cluster
Retrieval-Augmented Systems
Indexing, chunking, and grounding.
- 01 Why a Larger Context Window Does Not Remove the Need for an Index The arithmetic that separates a corpus from a prompt, what a retrieval step buys in tokens and memory, and the narrow cases where loading everything still wins.
- 02 Chunk Boundaries Decide What Can Be Retrieved How fixed-size, sentence, and structural chunking place boundaries, what overlap recovers and what it duplicates, and why the split point is an index-time commitment.
- 03 Approximate Nearest Neighbour Indexes and the Recall They Trade Away How graph, cluster, and hash indexes reduce a linear scan to a bounded walk, which knob controls recall, and why recall is a property of the query distribution rather than the index.
- 04 Similarity Metrics and the Normalisation They Require Why cosine, inner product, and Euclidean distance rank differently, what L2 normalisation changes about a vector index, and which failures come from mixing conventions.
- 05 Hybrid Lexical and Dense Retrieval: Fusing Two Ranked Lists Why BM25 and embeddings fail on different queries, how reciprocal rank fusion combines their lists without comparable scores, and where the fusion still loses.
- 06 Cross-Encoder Reranking and the Cost of Reading Every Pair Why a reranker sees query and passage together, what that buys over a dual encoder, and how candidate depth turns a quality gain into a latency bill.
- 07 Metadata Filtering Inside Vector Search Why post-filtering a top-k list cannot satisfy a restrictive predicate, how pre-filtering collapses an index walk, and what the middle strategies actually cost.
- 08 Keeping a Vector Index Current Without Rebuilding It How segments, tombstones, and compaction let an index absorb edits, why deletion is more expensive than insertion, and where the visible-lag window comes from.
- 09 Span Attribution: Making Generated Claims Point at Real Text How offsets survive chunking, encoding, and decoding; what a citation can and cannot prove; and why quote verification is a string operation rather than a judgement.
- 10 Context Dilution: When More Retrieved Passages Lower Answer Quality Why irrelevant context degrades generation rather than merely failing to help, how position in the prompt changes attention, and what to cut before raising k.