A systematic mental model for embeddings, sparse and dense search, chunking, hybrid retrieval, reranking, provenance, evaluation, and failure analysis.

An LLM cannot answer from a document it never receives. Before discussing prompts, agents, or generation quality, a retrieval system must solve a more basic problem:

Given an information need, which small pieces of a large corpus should be placed in a limited context window, in what order, and with what evidence about their origin?

That question contains several different jobs. Treating all of them as “RAG” hides where failures occur.

User need
   ↓
Query interpretation / rewriting
   ↓
Candidate generation  ← sparse search, dense search, or both
   ↓
Filtering             ← tenant, product, version, permissions, dates
   ↓
Ranking               ← inexpensive retrieval scores
   ↓
Reranking             ← slower, more accurate pairwise relevance model
   ↓
Context assembly      ← deduplicate, expand, order, budget, attach provenance
   ↓
Answer generation     ← synthesize only from supplied evidence
   ↓
Grounding verification and citations

These stages must remain distinguishable in code, traces, and evaluation:

Stage	Question	Typical output
Candidate generation	What might be relevant?	50–1,000 IDs per retriever
Filtering	What is allowed and applicable?	A permitted subset
Ranking	Which candidates appear more relevant?	Ordered candidates with retrieval scores
Reranking	Which candidates best answer this exact query?	A more accurate top 5–20
Context assembly	What evidence should the LLM actually see?	Ordered, budgeted text with source metadata
Answer generation	What answer follows from that evidence?	Response with claim-to-source citations

If the correct passage never becomes a candidate, no reranker or prompt can recover it. If the correct passage is retrieved but the answer is wrong, retrieval may be healthy and generation may be the faulty stage. This separation is the central invariant of the entire system.

1. The first search system: exact lookup

The information need

Suppose a developer asks:

What does error code PAYMENT_4027 mean?

If the documentation has a row keyed by that exact code, the ideal operation is not semantic search. It is a dictionary lookup:

SELECT description, remediation
FROM error_codes
WHERE code = 'PAYMENT_4027';

With a B-tree or hash-like index, this is fast, deterministic, and precise.

Why naive lookup fails

Users rarely preserve the canonical key. They may ask:

“Why was this payment rejected as already captured?”
“What is 4027?”
“payment duplicate capture failure”

Exact lookup has almost perfect precision when the key is known and nearly zero recall when the wording changes.

The lesson

Use exact lookup whenever the information need contains a stable identifier: document ID, order number, API symbol, error code, product SKU, or canonical URL. Do not replace deterministic retrieval with probabilistic retrieval merely because embeddings are available.

Diagnosis checkpoint: List the query classes in your system that should bypass semantic search. If error codes currently go through a vector index, compare exact-match success, latency, and cost against dense retrieval.

2. From exact keys to keyword search

Now the user asks:

How do I rotate an API key without downtime?

There is no single identifier, but documents containing rotate, API, key, and perhaps downtime are promising.

A naive implementation scans every document:

matches = [doc for doc in documents if "rotate" in doc.text.lower()]

This is slow for a large corpus, ignores multiple terms, treats common and rare terms equally, and cannot rank strong matches above weak ones.

We need two things:

A structure that finds documents containing a term without scanning the corpus.
A score that estimates how strongly each document matches the query.

Inverted indexes

An inverted index reverses the document-to-words relationship.

Documents:

D1: rotate an API key safely
D2: API rate-limit policy
D3: zero-downtime credential rotation

Inverted index:

api       → D1, D2
key       → D1
rotate    → D1
rotation  → D3
downtime  → D3

Real indexes also store term frequency and often positions:

api → [(D1, frequency=1, positions=[4]),
       (D2, frequency=1, positions=[1])]

Candidate generation now touches posting lists for query terms instead of every document. Stemming or linguistic normalization may reduce rotate, rotating, and rotation to related lexemes. Stop-word removal may discard terms such as the that carry little retrieval value.

PostgreSQL represents normalized documents as tsvector, queries as tsquery, and commonly uses a GIN inverted index for full-text search. Its dictionaries normalize tokens and can remove stop words. PostgreSQL full-text search introduction and text-search indexes describe the current mechanics.

Term frequency: evidence within one document

If replication appears five times in a PostgreSQL replication guide and once in a broad database overview, the guide is probably more focused on the query.

Raw term frequency is:

$$ tf(t,d) = \text{count of term }t\text{ in document }d $$

But ten occurrences should not imply ten times the relevance. Repetition has diminishing returns. Useful scoring functions therefore saturate term frequency rather than growing linearly.

Inverse document frequency: evidence across the corpus

The term database may occur almost everywhere. wal_keep_size may occur in very few documents. Matching the rare term is stronger evidence.

For a corpus of $N$ documents and document frequency $df(t)$:

$$ idf(t) \approx \log\left(\frac{N}{df(t)}\right) $$

High $df$: common term, low discriminative value.
Low $df$: rare term, high discriminative value.

The key distinction is:

Term frequency asks how prominent the term is inside this document.
Document frequency asks how rare the term is across the collection.

BM25 intuition

BM25 combines three useful beliefs:

A document matching more query terms is usually better.
Repeated occurrences help, but with diminishing returns.
A match in a short, focused document is usually stronger than the same count in a very long document.

One common form is:

$$ \operatorname{BM25}(q,d)=\sum_{t\in q} idf(t)\cdot \frac{tf(t,d)(k_1+1)} {tf(t,d)+k_1\left(1-b+b\frac{|d|}{avgdl}\right)} $$

where:

$|d|$ is document length.
$avgdl$ is average document length.
$k_1$ controls term-frequency saturation.
$b$ controls length normalization.
$idf(t)$ makes rare matches more valuable.

Consider a corpus of 1,000 chunks:

database occurs in 800 chunks.
ivfflat.probes occurs in 5 chunks.

A chunk containing both should rank strongly because the rare technical term contributes far more evidence than database. A chunk that repeats database 30 times should not win solely through repetition.

BM25 is a ranking function, while an inverted index is an access structure. They are related but not identical. PostgreSQL's built-in ts_rank and ts_rank_cd are full-text ranking functions with lexical, positional, structural, and optional normalization signals; they should not be casually labeled “BM25.” PostgreSQL ranking documentation explains those functions.

Lexical retrieval excels at:

Exact identifiers and error codes.
Function names, commands, acronyms, and product names.
Rare terms.
Auditable term matching.
Fast retrieval through mature inverted indexes.

It struggles with:

Synonyms: terminate versus shut down.
Paraphrases: avoid service interruption versus without downtime.
Conceptual relations with little vocabulary overlap.
Ambiguous terms: Java the language versus Java the island.
Vocabulary mismatch between novice questions and expert documentation.

This is not a reason to discard lexical search. It is a reason to add another retrieval signal for a specific failure class.

Implementation checkpoint: Build a tiny inverted index from 20 technical paragraphs. Store posting lists, frequency, and positions. Compare Boolean matching, TF-IDF, and BM25 rankings for an exact identifier query, a common-term query, and a paraphrase.

3. Semantic similarity and embeddings

The failure that creates dense retrieval

Query:

How can I change credentials without interrupting traffic?

Relevant passage:

Rotate API keys with a dual-key overlap period to achieve zero downtime.

The query and passage describe the same operational idea but may share no high-value words. Lexical search sees weak evidence. We need a representation in which meaning-related texts can be near one another despite vocabulary mismatch.

What an embedding is

An embedding model maps an input into a vector of numbers:

$$ f(text) \rightarrow \mathbf{x}\in\mathbb{R}^n $$

For illustration:

"rotate credentials safely" → [0.12, -0.41, 0.77, ...]

Individual dimensions usually do not have stable human labels such as “security” or “downtime.” Meaning is distributed across the vector. The geometry is useful because the model was trained so that certain related inputs receive similar representations.

An embedding is therefore not:

A compressed copy from which the original text can be faithfully reconstructed.
A truth score.
Proof that two texts answer one another.
A universal representation independent of the model that produced it.

It is a learned representation optimized for some notion of similarity.

Vector spaces

A vector space gives us mathematical operations over representations. In two dimensions we can draw vectors; real embeddings may have hundreds or thousands of dimensions, but the same operations apply.

Suppose:

q = [1, 1]
a = [2, 2]     # same direction as q
b = [1, -1]    # perpendicular to q

The search system converts the query and every chunk using compatible embedding logic, computes a similarity or distance, and selects nearest candidates.

The space only has operational meaning under the model that shaped it. Vectors from different embedding models, or incompatible model versions, cannot normally be compared directly. A model migration therefore requires a new embedding column or a full re-embedding process, plus evaluation before cutover.

Cosine similarity

Cosine similarity measures the angle between vectors:

$$ \cos(\mathbf{q},\mathbf{d})= \frac{\mathbf{q}\cdot\mathbf{d}} {\lVert\mathbf{q}\rVert\lVert\mathbf{d}\rVert} $$

For $q=[1,1]$ and $a=[2,2]$, cosine similarity is 1 because they point in the same direction. Their magnitudes differ, but their orientation does not.

Strength: useful when direction carries the desired semantic signal and magnitude is not meant to dominate.

Blind spot: it discards magnitude. That is correct only if magnitude is irrelevant or vectors are intentionally normalized.

Dot product

The dot product is:

$$ \mathbf{q}\cdot\mathbf{d}=\sum_i q_i d_i $$

It depends on both angle and magnitude:

$$ \mathbf{q}\cdot\mathbf{d}= \lVert\mathbf{q}\rVert\lVert\mathbf{d}\rVert\cos(\theta) $$

If embeddings are unit-normalized, dot product and cosine similarity produce the same ranking. If not, high-magnitude vectors may score higher even when their directions are less aligned.

Euclidean distance

Euclidean or L2 distance measures straight-line separation:

$$ d(\mathbf{q},\mathbf{d})= \sqrt{\sum_i(q_i-d_i)^2} $$

Lower is better. For unit-normalized vectors:

$$ \lVert\mathbf{q}-\mathbf{d}\rVert^2=2-2\cos(\mathbf{q},\mathbf{d}) $$

So cosine and Euclidean distance induce the same ordering when both vectors are normalized. Without normalization, they represent different beliefs.

Choosing the similarity measure

Do not choose by habit. Choose the metric the embedding model was trained and documented to use, then preserve any required normalization.

Measure	Better value	Magnitude matters?	Typical use
Cosine similarity	Higher	No, after normalization	Direction-based semantic similarity
Dot product	Higher	Yes, unless normalized	Models trained for inner-product retrieval
Euclidean distance	Lower	Yes	Models/indexes trained around L2 geometry

In pgvector, the commonly used operators are <=> for cosine distance, <#> for negative inner product, and <-> for L2 distance. The negative inner-product operator exists because PostgreSQL indexes support ascending scans. The pgvector documentation is the authoritative reference for current operator classes and index options.

Embedding models are retrieval components, not commodities

Embedding models differ in:

Training objective and data.
Supported languages.
Code versus prose capability.
Input length.
Dimensionality.
Query/document prefixes or task instructions.
Whether vectors are normalized.
Domain coverage.
Latency, memory, and price.

A larger model is not automatically better for your corpus. A code-focused model may outperform a general model on API documentation. A multilingual model may be necessary when queries and documents use different languages. The only defensible choice is comparative evaluation on representative queries.

Dense retrieval

Dense retrieval works as follows:

Embed each retrievable unit offline.
Embed the incoming query with the compatible query pathway.
Search for nearest vectors.
Return those chunks as candidates.

query_vector = embed_query(query)
candidates = vector_store.nearest(
    vector=query_vector,
    metric="cosine",
    limit=100,
)

Dense retrieval handles paraphrases and conceptual similarity well. Its blind spots include:

Exact rare identifiers may be diluted.
Nearby does not guarantee answer-bearing.
Similar topic is not the same as correct applicability.
Negation and small numeric differences may be missed.
Old and new versions can be semantically close.
Domain language outside training data can be represented poorly.

Dense retrieval is another candidate generator, not a complete retrieval system.

Diagnosis checkpoint: Create query-passage pairs where relevance depends on paraphrase, identifiers, negation, version, and numbers. Compare cosine, dot-product, and L2 rankings with and without normalization. Explain each ranking change geometrically.

4. Exact nearest neighbours and why approximation exists

With 10 million chunks of 1,536 dimensions, exact search compares the query with every vector. That is roughly 15.36 billion dimension-level operations per query before implementation optimizations. Exact search maximizes retrieval fidelity but may violate latency or throughput requirements.

Approximate nearest-neighbour search, or ANN, deliberately trades some recall for speed and scale:

Search a carefully chosen subset of vectors that is likely to contain the true nearest neighbours.

The word approximate matters. The ANN index may fail to return a vector that exact search would have placed in the top $k$.

Vector indexes

Two useful mental models are common.

IVFFlat: search nearby clusters

Partition vectors around learned centroids.
At query time, find the nearest centroids.
Search vectors in only those clusters.

More probed clusters improve recall and increase latency. Too few probes can miss relevant vectors near cluster boundaries. The index must be trained on representative data; data drift can degrade its partitioning.

HNSW: navigate a proximity graph

HNSW creates a multilayer graph. Upper layers provide long jumps; lower layers provide fine local search. Query-time search navigates from broad regions toward close neighbours.

Increasing the search breadth generally improves recall at the cost of latency. HNSW often offers a strong latency-recall trade-off but consumes extra memory and has meaningful build/update costs.

The correct ANN experiment

ANN recall is not the same as business retrieval recall. First compare ANN output against exact vector search:

$$ ANNRecall@k = \frac{|ANN_k(q)\cap Exact_k(q)|}{k} $$

Then separately compare retrieved results against human relevance labels. An ANN index can reproduce exact vector neighbours perfectly while the embedding model itself retrieves irrelevant passages.

Filters complicate ANN

Suppose vector search finds 40 approximate neighbours and a product_version = 17 filter is applied afterward. If only 10% belong to version 17, roughly four may survive. The issue is neither the embedding nor the requested LIMIT 10; it is the interaction between approximate traversal and filtering.

Current pgvector supports iterative scans that can continue scanning until enough filtered results are found or a configured limit is reached. It also recommends considering partial indexes or partitioning for suitable filter patterns. See pgvector filtering and iterative scans.

Never assume LIMIT 10 means ten relevant filtered results will be available. Trace pre-filter candidate counts, post-filter counts, index settings, and whether the target appeared under exact search.

Implementation checkpoint: On at least 100,000 embeddings, compare exact search, HNSW, and IVFFlat. Plot latency against ANN recall while varying search breadth, index parameters, and filter selectivity.

5. The retrievable unit: chunking

A 200-page manual cannot be placed into the context for every question. Embedding the whole manual creates one vector that mixes many topics. Returning the whole manual wastes context. Splitting it into independent sentences removes necessary surrounding information.

Chunking chooses the unit that candidate generation can retrieve.

Why chunk size creates a precision-recall trade-off

Chunks that are too large:

Mix unrelated topics into one vector.
Lower retrieval specificity.
Consume more context tokens.
Cause a relevant sentence to be buried inside noise.
Increase reranking and generation cost.

Chunks that are too small:

Lose definitions, conditions, and referents.
Produce fragments such as “This is disabled by default” without saying what “this” means.
Split procedures across several candidates.
Increase the number of indexed rows.
Make citations less intelligible.

There is no universal optimal token count. The correct unit is the smallest self-contained passage capable of answering a likely information need.

Structure-aware chunking

Prefer document boundaries with meaning:

Heading and subsection.
Paragraph.
List or procedure.
Code block plus its explanation.
Table with its title and headers.
API symbol or class.

Then enforce a maximum token budget by splitting oversized sections. Preserve heading paths so the chunk remains interpretable:

PostgreSQL Administration > Replication > Standby Configuration

[chunk text]

Chunk overlap

Overlap repeats some boundary text in adjacent chunks so an answer spanning a split remains retrievable.

It helps when:

A sentence depends on the previous paragraph.
A procedure crosses an arbitrary size boundary.
Boundary placement is imperfect.

It hurts when:

Duplicate chunks dominate top results.
Context contains the same statement several times.
Index size and embedding cost grow.
Metrics look inflated because near-duplicates occupy multiple relevant ranks.

Overlap is compensation for uncertain boundaries, not a default quality switch. Structure-aware splitting often needs less overlap than fixed windows.

Metadata

Text alone is insufficient. Store the information required to interpret, filter, cite, refresh, and authorize a chunk:

{
  "chunk_id": "pg18-repl-0042",
  "document_id": "postgresql-18-admin",
  "parent_id": "pg18-repl-section-7",
  "title": "Standby Server Settings",
  "heading_path": ["Replication", "Standby Configuration"],
  "product": "postgresql",
  "version": "18",
  "language": "en",
  "source_uri": "...",
  "content_hash": "...",
  "valid_from": "2026-08-01",
  "access_scope": "public",
  "start_offset": 18840,
  "end_offset": 20112
}

Metadata is not decoration. It supports security boundaries, applicability rules, lineage, deduplication, incremental re-indexing, and citations.

Filtering

A query may be semantically closest to the wrong product version, tenant, language, or permission scope. Filtering enforces constraints that similarity scores should not be expected to learn.

Examples:

tenant_id = current_tenant
access_scope ∈ authorized_scopes
product = 'postgresql'
version = '18'
valid_from <= now < valid_to

Hard eligibility constraints should normally be applied before final ranking, and authorization must never be delegated to the LLM. Whether a specific vector index applies filters during or after traversal is an implementation detail with major recall consequences; measure it.

Design checkpoint: Take three real technical documents. Produce fixed-window, paragraph, and heading-aware chunks. For 20 queries, label which chunks are minimally sufficient and compare recall, duplicate rate, and context tokens.

6. The user's query is not always the best retrieval query

Query rewriting

A user may ask:

After the upgrade, replicas fall behind whenever writes spike. What should I check?

The documentation may use terms such as replication lag, WAL generation, WAL receiver, network throughput, and replay delay.

A query rewriter can produce a search-oriented representation:

{
  "original": "After the upgrade, replicas fall behind whenever writes spike. What should I check?",
  "lexical_query": "replication lag WAL generation receiver replay delay upgrade",
  "semantic_query": "diagnose PostgreSQL replica lag correlated with high write throughput after version upgrade",
  "filters": {"product": "postgresql"},
  "unresolved": ["target version"]
}

Rewriting can expand acronyms, add domain terminology, remove conversational filler, infer explicit filters, or split compound questions.

Its risk is semantic drift. A rewriter can add an unsupported assumption, remove a crucial identifier, or make every query generic. Always retain the original query, trace the rewritten query, and evaluate both against the same relevance judgments.

Multi-query retrieval

One query representation expresses only one interpretation. Multi-query retrieval issues several intentionally different searches:

Q1: PostgreSQL replica lag during high write throughput
Q2: WAL generation exceeds standby replay capacity
Q3: post-upgrade replication performance regression

The union improves recall when the original phrasing is uncertain. It also increases latency, cost, duplicates, and irrelevant candidates. Diversity matters: five paraphrases that return the same set add little value.

Multi-query retrieval is justified when:

Queries are ambiguous or underspecified.
Recall is more important than first-stage precision.
The corpus uses several vocabularies.
A later reranker can absorb the larger candidate set.

It is not automatically useful for exact codes or well-formed API names.

Diagnosis checkpoint: Build a set of ambiguous and explicit queries. Measure the marginal recall gained by each additional rewrite, the candidate overlap between rewrites, latency, and reranker load.

7. Hybrid search: combine different evidence, not fashionable components

Sparse search fails on vocabulary mismatch. Dense search fails on exact identifiers, subtle constraints, and sometimes versions or numbers. When the failure sets are complementary, combining them is justified.

Candidate generation

Run both retrievers:

Sparse top 100: exact terms, identifiers, rare vocabulary
Dense top 100: paraphrases, concepts, semantic neighbours
Union: up to 200 candidates before deduplication

Hybrid retrieval improves recall only if each branch contributes useful candidates that the other misses. Measure unique relevant gains:

relevant from sparse only
relevant from dense only
relevant from both
relevant from neither

Why raw score addition is usually invalid

A sparse score of 12.4 and cosine similarity of 0.78 do not share units, distributions, or calibration. Even scores from the same retriever can shift when the corpus or query length changes.

This is mathematically unjustified:

$$ score = 0.5\cdot BM25 + 0.5\cdot cosine $$

unless both inputs have first been made comparable and the weights have been tuned on labeled data.

Score normalization

Possible approaches include:

Min-max within one result list

$$ s' = \frac{s-s_{min}}{s_{max}-s_{min}} $$

Simple, but unstable when the list has outliers or a narrow score range.

Z-score normalization

$$ s' = \frac{s-\mu}{\sigma} $$

Uses the list distribution but assumes that mean and variance are meaningful and reasonably stable.

Calibration on labeled data

Learn how each raw score maps to probability of relevance. This is more principled but requires enough representative labels and monitoring for drift.

After normalization or calibration:

$$ hybrid(d)=\alpha s'{sparse}(d)+(1-\alpha)s'{dense}(d) $$

Tune $\alpha$ rather than declaring 0.5 fair. Equality of coefficients does not imply equality of influence.

Rank fusion

Rank fusion avoids direct comparison of raw scores. Reciprocal Rank Fusion is:

$$ RRF(d)=\sum_{r\in retrievers}\frac{1}{k+rank_r(d)} $$

If a document ranks 2nd in sparse search and 5th in dense search, with $k=60$:

$$ RRF(d)=\frac{1}{62}+\frac{1}{65} $$

A candidate found by only one retriever still receives a score; one ranked highly by both gets reinforced. RRF is robust when score scales differ, although it discards information about score margins. The constant $k$ controls how sharply early ranks are favored.

Filtering, ranking, and fusion order

Do not let unauthorized candidates influence downstream results. A defensible order is:

Construct authorization and applicability filters.
Generate sparse and dense candidates within those constraints where possible.
Enforce filters again as a safety boundary.
Deduplicate by canonical chunk or content hash.
Fuse rankings.
Pass a bounded depth to the reranker.

Implementation checkpoint: For your labeled set, compare sparse, dense, normalized weighted fusion, and RRF. Report recall@50, MRR@10, NDCG@10, latency, and unique relevant contribution by branch.

8. Reranking: spend more computation on fewer pairs

First-stage retrievers must compare a query with a large corpus, so their scoring is intentionally efficient. They may retrieve topically related but non-answer-bearing passages.

Query:

Can an HNSW index return fewer than ten rows after a category filter?

Candidate A discusses HNSW generally. Candidate B specifically explains post-index filtering and iterative scans. Both are semantically related; B answers the exact question.

Reranking applies a more accurate scorer to only the candidate set:

200 hybrid candidates
      ↓
cross-encoder scores 50 selected candidates
      ↓
top 8 for context assembly

Cross-encoders

A bi-encoder embeds query and document separately:

$$ score(q,d)=similarity(E_q(q),E_d(d)) $$

Document embeddings can be precomputed, which makes corpus-scale search practical.

A cross-encoder processes the pair jointly:

$$ score(q,d)=CrossEncoder([q;d]) $$

Because tokens from the query and passage can interact during scoring, a cross-encoder can detect exact answer relationships, negation, and conditions better. But document representations cannot generally be precomputed independently, so scoring every corpus document would be too expensive.

Reranking depth

If the reranker sees only the top 10 first-stage candidates, it cannot rescue a relevant passage at rank 11. If it sees 500, latency and cost may be excessive.

Tune depth as a controlled variable:

First-stage recall at depth $n$ establishes the ceiling.
Reranker NDCG/MRR measures reordering quality.
End-to-end context recall shows whether answer-bearing evidence survives.
Latency and cost show the price of improvement.

An LLM can also rerank, but it introduces greater cost, nondeterminism, prompt sensitivity, and possible position bias. Treat it as a measured component, not an unquestioned judge.

Diagnosis checkpoint: For each reranking depth in {10, 25, 50, 100}, report how often the best labeled passage reaches the top 5, along with p50/p95 latency and cost.

9. Parent-child retrieval and contextual chunking

The granularity conflict

Small chunks are easy to match precisely. Larger sections contain the context needed to understand and answer. One unit need not serve both purposes.

Parent-child retrieval

Index small child chunks but return a larger parent section:

Parent: "HNSW filtering and iterative scans" (1,200 tokens)
  ├─ Child 1: filtering occurs after approximate scan
  ├─ Child 2: strict iterative scans
  └─ Child 3: relaxed ordering and limits

The child is the candidate-generation unit. The parent is a possible context-assembly unit.

Blind spots:

Returning every parent can waste tokens.
Several winning children may map to the same parent.
A parent may contain unrelated details.
Expansion can exceed the context budget.

Deduplicate parents, cap expansion, or retrieve a child plus adjacent siblings rather than the entire section.

Contextual chunking

A raw chunk may say:

It is disabled by default and should be enabled only for selective filters.

It lacks the subject. Contextual chunking enriches its retrievable text with concise document context:

Document: pgvector index tuning
Section: Iterative HNSW scans under metadata filters
Context: This passage explains when iterative ANN scanning is needed because
post-index filters can leave too few results.

Original passage: It is disabled by default ...

The contextual prefix is included for retrieval, while the original text and provenance remain separately stored.

This improves self-containment but can fail if generated context hallucinates, injects generic vocabulary into every chunk, or overwhelms exact content. Context generation must be traceable, versioned, and evaluated against raw chunks. Never replace the source text with generated context.

Design checkpoint: Compare child-only, parent return, adjacent-window expansion, and contextualized chunks. Measure answer-bearing recall per 1,000 context tokens, not recall alone.

10. Context assembly is a retrieval stage of its own

Even a good ranked list is not yet an LLM context. Context assembly decides what survives the token budget and how evidence is represented.

It should:

Enforce authorization and applicability again.
Remove exact and near duplicates.
Expand parents or neighbours only when needed.
Prefer answer-bearing passages over merely topical ones.
Preserve titles, headings, source IDs, and offsets.
Order evidence deliberately.
Fit the token budget without cutting away critical qualifiers.

<source id="S3"
        document_id="pgvector-readme"
        section="Filtering"
        version="0.8+"
        uri="https://github.com/pgvector/pgvector/blob/master/README.md#filtering">
  [verbatim source passage]
</source>

Candidate scores are not truth probabilities. Do not tell the generator that a chunk is “92% true” because cosine similarity is 0.92.

Citations

A citation is a user-facing pointer connecting an answer claim to supporting evidence. A trustworthy citation must be:

Entailed: the cited passage actually supports the claim.
Precise: it points to the relevant section or span, not merely a homepage.
Stable: it uses a durable identifier or version when possible.
Complete: important factual claims are covered.
Faithful: the generator cannot invent source IDs.

The safest pattern is to assign source IDs during context assembly and constrain the generator to cite only those IDs. After generation, validate that every cited ID exists and optionally run claim-evidence entailment checks.

Provenance

Citations are the visible end of a deeper lineage chain:

answer claim
  → context source ID
  → retrieved chunk ID and offsets
  → parent document ID and version
  → source URI / repository commit / ingestion timestamp
  → parser version
  → chunker configuration
  → contextualizer version
  → embedding model and version

Provenance makes the system debuggable and reproducible. Without it, you cannot answer:

Which source version produced this statement?
Was the source later replaced?
Did a parser omit a table?
Which embedding model indexed the chunk?
Why did the same query change after deployment?

Implementation checkpoint: Given an answer with five claims, produce a claim-to-source matrix. Mark each claim as supported, partially supported, unsupported, or contradicted. Reject citations that point to a relevant document but not to evidence for the claim.

11. Evaluate retrieval before evaluating generation

Suppose the final answer is wrong. At least four very different failures are possible:

The corpus lacks the answer.
The corpus has it, but candidate generation misses it.
It is retrieved, but ranking or context assembly removes it.
The LLM sees it and still answers incorrectly.

End-to-end answer accuracy cannot identify which occurred. Build a retrieval dataset and evaluate each boundary independently.

Retrieval datasets

Each evaluation example should contain more than a question and a single “correct chunk”:

{
  "query_id": "q-017",
  "query": "Why can filtered HNSW search return fewer rows than LIMIT?",
  "required_filters": {"product": "pgvector"},
  "relevance": [
    {"chunk_id": "c-88", "grade": 3, "reason": "directly explains post-scan filtering"},
    {"chunk_id": "c-89", "grade": 2, "reason": "explains iterative scan remedy"},
    {"chunk_id": "c-11", "grade": 1, "reason": "general HNSW background"}
  ],
  "answerable": true,
  "query_type": "diagnosis",
  "difficulty": "hard",
  "notes": "requires distinguishing SQL LIMIT from ANN candidate breadth"
}

Use graded relevance because a direct answer, useful supporting passage, and merely topical passage are not equivalent.

Include query categories such as:

Exact identifiers.
Paraphrases.
Multi-hop or multi-passage needs.
Version-specific questions.
Numeric or negation-sensitive questions.
Ambiguous queries.
Unanswerable questions.
Permission-sensitive queries.
Tables, code, procedures, and definitions.

Avoid building the set solely from chunks by asking an LLM to generate obvious questions. That produces unrealistic vocabulary overlap and overestimates retrieval quality. Include real search logs, expert-authored cases, observed failures, and adversarial variants. Remove private data and sample deliberately rather than blindly mirroring traffic frequency.

Precision@k

Precision asks what fraction of retrieved results are relevant:

$$ Precision@k=\frac{#\text{ relevant results in top }k}{k} $$

If 3 of the top 5 are relevant, precision@5 is 0.6.

Precision matters when context space is scarce or irrelevant chunks distract generation. Its blind spot is that it does not say how many relevant items were missed.

Recall@k

Recall asks what fraction of all known relevant results were retrieved:

$$ Recall@k=\frac{#\text{ relevant results in top }k} {#\text{ known relevant results}} $$

If four chunks are labeled relevant and three appear in the top 10, recall@10 is 0.75.

In RAG, an alternative hit rate or answer-bearing recall may be more appropriate when any one of several equivalent passages is sufficient:

$$ Hit@k = \mathbb{1}[\text{at least one sufficient passage appears in top }k] $$

Recall depends on judgment completeness. Unlabeled relevant passages can make a good system appear wrong. Pool candidates from multiple retrievers and have humans judge the pooled set to improve label coverage.

Mean Reciprocal Rank

Reciprocal rank rewards placing the first relevant result early:

$$ RR(q)=\frac{1}{rank\ of\ first\ relevant\ result} $$

Across queries:

$$ MRR=\frac{1}{|Q|}\sum_{q\in Q}RR(q) $$

First relevant result at ranks 1, 2, and 5 gives:

$$ MRR=\frac{1+1/2+1/5}{3}\approx0.567 $$

MRR is useful when one correct result is sufficient. It ignores the quality and order of later relevant results.

NDCG intuition

Normalized Discounted Cumulative Gain handles graded relevance and rewards strong results near the top.

$$ DCG@k=\sum_{i=1}^{k}\frac{2^{rel_i}-1}{\log_2(i+1)} $$

Compute the ideal DCG by sorting results by true relevance, then normalize:

$$ NDCG@k=\frac{DCG@k}{IDCG@k} $$

The score ranges from 0 to 1 when relevance labels are nonnegative. It asks: “How close is this ordering to the best possible ordering?” Unlike MRR, it values multiple relevant results and their grades.

Hard negatives

Random irrelevant chunks are too easy. Hard negatives look plausible but are wrong:

Correct topic, wrong version.
Same API name in another product.
Similar procedure with a different prerequisite.
Passage that states the opposite through negation.
Deprecated documentation.
Same error symptom with a different cause.
Chunk mentioning all query words but not answering the question.

Hard negatives expose whether the system understands applicability rather than topic resemblance. Mine them from high-ranked false positives and add them to both evaluation and, where appropriate, model training.

Answer-grounding evaluation

Retrieval relevance and answer grounding are related but distinct.

Retrieval relevance: Did the system supply evidence capable of answering the question?
Groundedness/faithfulness: Are answer claims supported by the supplied evidence?
Answer correctness: Is the answer correct according to the reference or expert judgment?
Citation correctness: Does each citation support the attached claim?
Citation completeness: Are all important externally verifiable claims cited?

A grounded answer can still be wrong if the source is obsolete. A correct answer can be ungrounded if the model supplied it from parametric memory rather than the retrieved evidence. Evaluate both.

Report metrics by slice

An average can conceal severe failures. Slice results by:

Query type.
Product and version.
Document format.
Head versus tail traffic.
Easy versus hard.
Exact versus paraphrased.
Single-passage versus multi-passage.
Filter selectivity.
Answerable versus unanswerable.

Always attach confidence intervals or bootstrap intervals when comparing close systems. A 0.7-point NDCG improvement on 30 queries may be noise.

Evaluation checkpoint: Create at least 100 judged queries with pooled candidates. Implement precision@k, recall@k, hit@k, MRR, and NDCG yourself before relying on a library. Unit-test each metric with tiny rankings whose answers you can calculate by hand.

12. Failure analysis: turn a bad answer into a stage-localized diagnosis

“The RAG system gave a bad answer” is not a useful bug report. Use a failure taxonomy.

Corpus and ingestion failures

The answer is absent.
The correct version was never ingested.
A parser dropped tables, code, headings, or footnotes.
OCR corrupted key terms.
Duplicate or stale documents dominate.

Evidence: inspect source coverage, parser output, content hashes, and version metadata.

Chunking failures

The answer spans chunks.
The chunk contains an unresolved pronoun.
A table was separated from its headers.
A code block was separated from explanation.
Large chunks mix topics.

Evidence: view source boundaries and minimally sufficient labeled spans.

Query-understanding failures

An acronym was not expanded.
A rewrite changed intent.
A compound query was not decomposed.
A required version filter was not inferred or requested.

Evidence: compare original query, rewrites, extracted filters, and per-query candidate sets.

Candidate-generation failures

Sparse search missed a paraphrase.
Dense search missed an exact identifier.
ANN missed an exact-vector neighbour.
The candidate depth was too shallow.
Metadata filtering eliminated the answer.

Evidence: compare sparse, dense, hybrid, exact-vector, pre-filter, and post-filter results.

Ranking and reranking failures

Raw incompatible scores were added.
A relevant passage existed at rank 60 but reranking depth was 50.
The reranker preferred topical text over answer-bearing text.
Positional bias affected LLM reranking.

Evidence: preserve every stage's rank, score, and model/version.

Context-assembly failures

Deduplication removed the wrong passage.
Parent expansion consumed the budget.
A qualifier was truncated.
Evidence order buried the best source.
Source IDs were detached from text.

Evidence: record selected and dropped chunks with explicit reasons and token counts.

Generation and grounding failures

The LLM ignored clear evidence.
It combined incompatible versions.
It asserted more than the sources support.
It invented a citation.
It should have abstained.

Evidence: evaluate the generated answer against the exact assembled context, not against the corpus at large.

A practical diagnostic sequence

For every failed query, ask in this order:

Is the needed information in the corpus?
Is it represented correctly after parsing?
Is there a minimally sufficient chunk or retrievable child?
Does sparse retrieval find it, and at what rank?
Does exact dense retrieval find it?
Does ANN reproduce exact dense retrieval closely enough?
Do filters preserve it?
Does hybrid fusion improve or demote it?
Is it inside reranking depth?
Does reranking move it into the context set?
Does context assembly preserve all necessary evidence?
Given that exact context, does generation answer faithfully?

This sequence replaces prompt guessing with evidence.

Practical project: an evaluated technical-document retrieval system

The project is not “build a chatbot.” It is “build a retrieval laboratory with an optional answer generator.” The primary output is comparative evidence about which retrieval design works and why.

13. System boundaries

Ingestion path
Documents → parser → structured blocks → chunker → contextualizer (optional)
          → embeddings + tsvector → PostgreSQL/pgvector

Online path
Query → filter extraction + rewrites
      → sparse candidates ┐
                          ├→ fusion → reranker → context assembler
      → dense candidates  ┘                       ↓
                                             evidence package
                                                  ↓
                                         answer + citations

Evaluation path
Dataset → experiment runner → retrieval metrics + traces + latency/cost
        → experiment registry → comparison dashboard

Keep the answer generator behind an interface so all retrieval experiments can run without it.

14. PostgreSQL schema

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE documents (
    id              uuid PRIMARY KEY,
    canonical_uri   text NOT NULL,
    title           text NOT NULL,
    product         text,
    version         text,
    language        text NOT NULL DEFAULT 'en',
    access_scope    text NOT NULL,
    content_hash    text NOT NULL,
    source_updated_at timestamptz,
    ingested_at     timestamptz NOT NULL DEFAULT now(),
    parser_version  text NOT NULL,
    UNIQUE (canonical_uri, content_hash)
);

CREATE TABLE chunks (
    id                uuid PRIMARY KEY,
    document_id       uuid NOT NULL REFERENCES documents(id),
    parent_chunk_id   uuid REFERENCES chunks(id),
    ordinal           integer NOT NULL,
    heading_path      text[] NOT NULL DEFAULT '{}',
    raw_text          text NOT NULL,
    retrieval_text    text NOT NULL,
    token_count       integer NOT NULL,
    start_offset      integer,
    end_offset        integer,
    chunk_hash        text NOT NULL,
    chunker_version   text NOT NULL,
    contextualizer_version text,
    embedding_model   text NOT NULL,
    embedding         vector(1536), -- match the selected model
    search_vector     tsvector GENERATED ALWAYS AS (
        setweight(to_tsvector('english', coalesce(array_to_string(heading_path, ' '), '')), 'A') ||
        setweight(to_tsvector('english', coalesce(retrieval_text, '')), 'B')
    ) STORED,
    UNIQUE(document_id, ordinal, chunker_version)
);

CREATE INDEX chunks_search_gin
ON chunks USING GIN (search_vector);

CREATE INDEX chunks_embedding_hnsw
ON chunks USING hnsw (embedding vector_cosine_ops);

CREATE INDEX chunks_document_idx ON chunks(document_id);
CREATE INDEX documents_product_version_idx ON documents(product, version);

PostgreSQL documents the generated tsvector plus GIN pattern in its tables and indexes guide. Use field weighting deliberately; a heading match may deserve more influence than a body match.

For an embedding migration, add a new embedding column/table keyed by model version. Do not overwrite the old vectors before comparative evaluation and rollback are possible.

15. Ingestion pipeline

For each document:

Fetch and hash the source.
Parse into typed blocks: heading, paragraph, list, code, table.
Preserve source offsets or page/section locators.
Build structure-aware chunks.
Optionally create a contextual prefix while retaining raw text.
Count tokens with the intended tokenizer.
Embed in deterministic batches.
Write chunks and provenance transactionally.
Mark the document version active only after all chunks succeed.
Run ingestion quality checks.

Quality checks should catch:

Empty or extremely large chunks.
Tables without headers.
Broken Unicode or OCR.
Duplicate chunk hashes.
Missing embeddings.
Embedding dimensionality mismatch.
Missing source locators.
Sudden document/chunk-count changes.

16. Sparse and dense candidate queries

Sparse retrieval

WITH q AS (
  SELECT websearch_to_tsquery('english', :lexical_query) AS query
)
SELECT
  c.id,
  ts_rank_cd(c.search_vector, q.query, 32) AS sparse_score
FROM chunks c
JOIN documents d ON d.id = c.document_id
CROSS JOIN q
WHERE c.search_vector @@ q.query
  AND d.product = :product
  AND (:version IS NULL OR d.version = :version)
  AND d.access_scope = ANY(:allowed_scopes)
ORDER BY sparse_score DESC
LIMIT :sparse_k;

Dense retrieval

SELECT
  c.id,
  c.embedding <=> :query_embedding AS cosine_distance
FROM chunks c
JOIN documents d ON d.id = c.document_id
WHERE d.product = :product
  AND (:version IS NULL OR d.version = :version)
  AND d.access_scope = ANY(:allowed_scopes)
ORDER BY c.embedding <=> :query_embedding
LIMIT :dense_k;

Convert distance to a clearly named similarity only if needed:

cosine_similarity = 1 - cosine_distance

Do not name a distance column score without specifying direction. Every trace should record whether higher or lower is better.

17. Fusion and reranking service

Use canonical chunk IDs for deduplication. A simple RRF implementation:

from collections import defaultdict

def reciprocal_rank_fusion(result_lists, k=60):
    fused = defaultdict(float)
    evidence = defaultdict(list)

    for retriever_name, ids in result_lists.items():
        for rank, chunk_id in enumerate(ids, start=1):
            contribution = 1.0 / (k + rank)
            fused[chunk_id] += contribution
            evidence[chunk_id].append({
                "retriever": retriever_name,
                "rank": rank,
                "contribution": contribution,
            })

    ranked = sorted(fused, key=fused.get, reverse=True)
    return ranked, evidence

Pass the top rerank_depth candidates to a cross-encoder. Store its model version, input truncation, score, and latency. Then take the top candidates that fit the context policy.

18. Retrieval trace schema

Every query should yield a trace independent of answer generation:

{
  "trace_id": "rt-...",
  "query": {
    "original": "...",
    "rewrites": ["..."],
    "filters": {"product": "postgresql", "version": "18"}
  },
  "configuration": {
    "chunker": "heading-v3-512",
    "embedding_model": "model-name@version",
    "dense_metric": "cosine",
    "dense_k": 100,
    "sparse_k": 100,
    "fusion": "rrf@60",
    "reranker": "model-name@version",
    "rerank_depth": 50,
    "context_budget_tokens": 5000
  },
  "stages": {
    "sparse": [{"chunk_id": "...", "rank": 1, "score": 0.71}],
    "dense": [{"chunk_id": "...", "rank": 1, "distance": 0.18}],
    "fused": [{"chunk_id": "...", "rank": 1, "score": 0.032}],
    "reranked": [{"chunk_id": "...", "rank": 1, "score": 8.4}],
    "selected_context": [
      {"chunk_id": "...", "tokens": 311, "source_id": "S1"}
    ],
    "dropped": [
      {"chunk_id": "...", "reason": "near_duplicate"}
    ]
  },
  "timing_ms": {
    "rewrite": 20,
    "sparse": 12,
    "dense": 18,
    "fusion": 1,
    "rerank": 95,
    "assembly": 2
  }
}

Traces make every metric explainable. A dashboard without drill-down traces encourages superstition.

19. Evaluation dataset and experiment runner

Create versioned tables or files for:

Query text and category.
Required filters.
Graded relevant chunk IDs or source spans.
Whether any one passage is sufficient.
Whether multiple evidence units are required.
Expected answer or claim checklist for later generation evaluation.
Annotator and disagreement status.

An experiment configuration should be immutable:

experiment_id: hybrid-rerank-042
corpus_snapshot: docs-2026-08-17
dataset_version: retrieval-eval-v4
chunking:
  strategy: heading_aware
  target_tokens: 512
  overlap_tokens: 64
embedding:
  model: model-name
  version: v2
retrieval:
  sparse_k: 100
  dense_k: 100
  fusion: rrf
  rrf_k: 60
reranking:
  model: reranker-name
  depth: 50
context:
  max_tokens: 5000

Persist per-query results, not only aggregate metrics. Aggregates tell you that something changed; per-query deltas tell you why.

20. Controlled experiments

Change one independent variable at a time or use a declared factorial design. Hold corpus snapshot, evaluation set, filters, and all other configuration constant.

Chunk size

Test, for example, 256, 512, and 1,024 target tokens.

Measure:

Hit@k and NDCG@k.
Minimally sufficient passage rate.
Context precision.
Tokens per selected context.
Duplicate rate.
End-to-end grounded answer quality.

Expected trade-off: larger chunks may improve completeness but reduce retrieval specificity and increase context cost.

Overlap

Test 0, 32, 64, and 128 tokens, but interpret relative to chunk size.

Measure boundary-query recall, near-duplicate rate, index size, and redundant context tokens. If overlap helps only because the splitter breaks semantic units, improve the splitter before accepting permanent duplication.

Embedding model

Re-embed the same chunks with each model. Keep metric and model-required formatting correct. Compare by query slice, especially code, prose, identifiers, and multilingual cases. Report embedding latency, storage, and cost as well as ranking quality.

Sparse versus dense retrieval

Run sparse-only and dense-only first. Their disagreement is the reason hybrid search may help. Use a Venn-style count of relevant hits: sparse-only, dense-only, both, neither.

Hybrid weighting or fusion

Compare RRF with calibrated or normalized weighted fusion. Sweep weights rather than testing only 0.5. Tune on a development set and report once on a held-out test set; otherwise the evaluation set becomes training data.

Reranking depth

Test depths such as 10, 25, 50, and 100. Plot top-5 relevance or NDCG against p95 latency and cost. The first-stage recall at each depth is the reranker's maximum possible rescue rate.

Experimental discipline

For every experiment record:

Hypothesis.
Independent variable.
Controlled variables.
Dataset and corpus versions.
Primary metric and guardrail metrics.
Per-slice outcomes.
Confidence interval.
Latency and cost.
Failure examples.
Decision and rollback condition.

Do not select a configuration merely because it wins one aggregate metric. A system that raises average NDCG while breaking exact error-code lookup may be unacceptable.

21. Comparison dashboard

The dashboard should answer decisions, not merely display charts.

Overview

Recall@5/10/50.
Hit@5/10.
MRR@10.
NDCG@10.
Context precision and tokens.
p50/p95/p99 latency by stage.
Cost per 1,000 queries.

Comparative views

Baseline versus challenger with absolute and relative delta.
Confidence interval on metric differences.
Quality-latency Pareto frontier.
Metrics by query category and filter selectivity.
Sparse-only/dense-only overlap.
Reranker promotions and demotions.
ANN recall versus exact vector search.

Failure explorer

For one query, show:

Original and rewritten queries.
Applied filters.
Sparse and dense ranked lists.
Fusion contributions.
Reranker changes.
Selected and dropped context.
Ground-truth judgments.
Generated answer and claim-level citations, if generation was run.

This is the most important view because it turns a score delta into an engineering explanation.

Reconstructing the complete mental model

A retrieval system is a sequence of lossy transformations.

The corpus limits what can be known from retrieval.
Parsing decides which source information survives ingestion.
Chunking defines the units that can be found.
Metadata and authorization define which units are applicable and allowed.
Sparse retrieval finds lexical evidence through inverted indexes and term statistics.
Dense retrieval finds geometric neighbours in a learned embedding space.
ANN indexes accelerate vector search by accepting a measurable recall trade-off.
Query rewriting and multi-query retrieval change the representation of the information need, with a risk of drift.
Hybrid fusion combines complementary candidate sets without pretending incompatible scores share units.
Reranking spends more computation to judge query-passage relevance precisely.
Parent-child expansion and contextual chunking reconcile matching granularity with explanatory completeness.
Context assembly deduplicates, budgets, orders, and attaches provenance.
Generation synthesizes from the evidence but can still ignore or distort it.
Citation and grounding checks connect claims back to exact source spans.
Evaluation and traces measure each boundary so failures can be localized.

The goal is not to maximize similarity. The goal is to maximize the probability that the context contains sufficient, applicable, authorized, nonredundant evidence for the user's information need—at an acceptable latency and cost—and to make every final claim traceable to that evidence.

Mastery gate

You have mastered this topic when you can produce the following without relying on framework vocabulary:

1. Explain lexical and semantic retrieval

Derive lexical retrieval from the need to avoid a corpus scan, explain inverted indexes, TF, IDF, BM25 saturation and length normalization, then explain vocabulary mismatch. Derive embeddings as learned representations that make semantic proximity searchable, while naming their failures on identifiers, versions, numbers, and applicability.

2. Choose a similarity measure

Explain cosine, dot product, and Euclidean distance geometrically and algebraically. State when normalization makes their rankings equivalent and choose the metric required by the embedding model rather than by convention.

3. Design chunks

Identify the smallest self-contained answer-bearing units in a real document. Defend boundaries, target size, overlap, parent relationships, metadata, and table/code handling using measured retrieval and context outcomes.

4. Explain vector indexing

Distinguish exact vector search from ANN. Explain IVFFlat as clustered search and HNSW as graph navigation. Quantify the latency-recall trade-off and show how filtering can reduce ANN results.

5. Build hybrid retrieval

Show the unique relevant contribution of sparse and dense candidate generators. Fuse them using justified normalization/calibration or rank fusion. Preserve per-retriever ranks and contributions in traces.

6. Separate retrieval and generation failures

Given a wrong answer, determine whether the source was absent, parsing failed, chunks were insufficient, candidates were missed, filters removed them, ranking demoted them, context assembly dropped them, or generation ignored them.

7. Design a retrieval evaluation dataset

Create representative, versioned queries with graded relevance, required filters, hard negatives, unanswerable cases, and realistic language. Explain label incompleteness and how pooled judging reduces it.

8. Interpret retrieval metrics

Calculate precision, recall, hit rate, reciprocal rank, MRR, DCG, and NDCG by hand on a small example. Choose metrics based on whether one or several relevant passages are required and inspect sliced results rather than trusting averages.

9. Diagnose poor retrieval

Use traces and exact-search baselines to locate the failing stage. Propose an intervention tied to evidence, then run a controlled experiment rather than changing chunking, prompts, embeddings, and reranking simultaneously.

10. Reconstruct the pipeline

Starting with an information need, describe every transformation through candidate generation, filtering, ranking, reranking, context assembly, generation, citations, and evaluation. For each boundary, state its input, output, scoring or decision rule, failure modes, trace fields, and metric.

If you can do all ten, you no longer understand RAG as “put documents in a vector database and ask an LLM.” You understand it as an evaluated information-retrieval system whose final consumer happens to be a language model.

Compact implementation sequence

Build in this order so that every added component answers an observed failure:

Establish exact lookup for stable identifiers.
Build a sparse PostgreSQL full-text baseline.
Create the evaluation dataset and metric implementation.
Add structure-aware chunks and provenance.
Add exact dense retrieval and compare it with sparse retrieval.
Add ANN only after exact-search latency requires it; measure ANN recall.
Add metadata and authorization filters; test selective-filter behavior.
Add hybrid fusion only if sparse and dense have complementary wins.
Add query rewriting only for query classes with demonstrated vocabulary or intent failures.
Add reranking only if relevant candidates exist but are poorly ordered.
Add parent-child expansion or contextual chunking only for granularity failures.
Assemble budgeted context with source IDs and provenance.
Add generation and citations.
Evaluate grounding and answer correctness separately from retrieval.
Ship the comparison dashboard and regression gate alongside the system.

Retrieval before generation is not merely an architectural ordering. It is an epistemic discipline: first prove that the evidence was available, found, allowed, ranked, and preserved; only then judge what the model did with it.