Mental model
Module 08: Combine exact words with meaning- Learn when lexical search and semantic search complement each other, and combine rankings without assuming one retriever is always best.
RAG & Knowledge Systems Engineering · Module 08 of 18
Builder
Keyword + Semantic Hybrid Search
Learn when lexical search and semantic search complement each other, and combine rankings without assuming one retriever is always best.
Engineering focus: Combine exact words with meaning
In plain English
What problem are we solving?
Learn when lexical search and semantic search complement each other, and combine rankings without assuming one retriever is always best.
You are not expected to know the engineering vocabulary before using the idea. Formal terminology comes after the mental model.
Running case · Evidence Desk
Improve the same system, module by module.
Carry one evidence-grounded knowledge assistant through all 18 modules. Preserve source IDs, retrieval evidence, evaluation cases, authority decisions and security assumptions so every improvement is measurable.
Mental model
Understand it before you optimize it.
Hybrid search uses two senses: exact-word hearing and semantic hearing. Fusion combines both candidate lists so each covers the other’s blind spots.
01Lexical ranking
02Dense ranking
03Normalize / fuse
04Deduplicate
05Evaluate
Remember
Hybrid retrieval is useful only when measured against representative queries.
Learning objectives
By the end of this module, you should be able to:
- Explain strengths of BM25/lexical and dense retrieval.
- Combine ranking signals using a simple fusion method.
- Choose hybrid retrieval based on evaluation results.
Lexical search is excellent for exact tokens
Error codes, product IDs, command names and legal phrases often benefit from keyword-oriented ranking.
Dense search handles paraphrase
Semantic retrieval can find related text even when the query and document use different words.
Hybrid reduces blind spots
Combining lexical and dense candidates often improves robustness across mixed query types.
Fusion needs measurement
Reciprocal rank fusion and weighted scoring are useful patterns, but tune them against representative queries.
Engineering depth
Engineering deep dive
- Evaluate lexical-heavy, semantic-heavy and mixed query cohorts separately.
- Reciprocal Rank Fusion is rank-based; weighted score fusion requires comparable score calibration.
- Deduplicate by stable document/chunk identity after fusion.
- Keep each retriever’s raw ranking for diagnosis.
Worked examples
Two retrievers, one fused rank
query: "ERR-104 token reset"
LEXICAL: A(ERR-104), C, B
DENSE: B(reset credentials), A, D
RRF: A, B, C, D- Exact identifiers help lexical search.
- Paraphrase helps dense search; fusion combines rank evidence.
Diagnostic lens
If this goes wrong, where do you look first?
Symptom
Exact identifiers work but paraphrases fail—or vice versa.
Likely causes
- Only lexical search is used
- Only dense search is used
- Fusion weights/ranks are poor
First checks
- Split evaluation by query type
- Inspect both rankings before fusion
- Check deduplication after fusion
Engineering principle
Hybrid search should address measured complementary failure modes.
Try it · core lab · account-free
Lab 08 — Keyword + Semantic Hybrid Search
Goal: Fuse two ranked lists with reciprocal rank fusion.
Before you start
- Core lab requires only Python 3 and local files unless otherwise stated.
- Running case: build the same evidence-grounded knowledge assistant across the course.
- Keep source IDs and intermediate evidence so each result is inspectable.
- Reflection: Write one sentence describing what this step proves and one sentence describing what it does not prove.
Run the baseline
python3 - <<'PY2'
lex=['docA','docC','docB']; dense=['docB','docA','docD']; scores={}
for ranking in (lex,dense):
for r,d in enumerate(ranking,1): scores[d]=scores.get(d,0)+1/(60+r)
print(sorted(scores.items(),key=lambda x:x[1],reverse=True))
PY2Inspect the evidence
Record the intermediate source IDs, ranks, metadata or state produced by the baseline. Explain which pipeline layer this evidence belongs to.
Split query types
Test one exact identifier query and one paraphrase query. Record which retriever wins each case before fusion.
Diagnose and restore
Hybrid search should address measured complementary failure modes. State the failing layer, change only that layer, then repeat the verification checks.
Write the engineering note
Record: hypothesis → evidence → change → observed result → remaining limitation. Keep this note with the running Evidence Desk project.
Verify
- Candidates from both retrievers contribute to the final ranking.
- Exact-token and paraphrase queries can be evaluated separately.
- The baseline and the deliberately broken case produce different, explainable evidence.
- You can name the pipeline layer that owns the failure and the control that repairs it.
Expected outcome
- A reproducible local result plus a short evidence note.
- A short before/after record showing the failure, diagnosis and repaired state.
Break / fix · diagnose
- Inspect the intermediate evidence before changing the model or prompt.
- If a command fails, preserve the error and classify which pipeline layer failed.
- Change one pipeline variable at a time and rerun the same test case.
- Do not accept a plausible answer as proof; verify the intermediate evidence and source identity.
Production view
Production RAG improves when every change can be traced to evidence at a specific pipeline layer. In production, retain the before/after evidence so the same failure can become a regression test.
Production checklist
What would an engineer verify before shipping this layer?
Lexical and dense rankings traceable
Fusion evaluated by query cohort
Deduplication stable
Avoid these traps
Common beginner and engineering mistakes
- Replacing keyword search everywhere with vectors.
- Hard-coding fusion weights without evaluation.
- Deduplicating sources incorrectly after fusion.
Knowledge check
Can you explain the idea and diagnose the system?
1When does lexical retrieval often shine?
Exact identifiers, names, commands and rare terms.
2What is a key dense-search strength?
Finding semantically related text despite wording differences.
3Why use hybrid search?
To combine complementary retrieval signals.
4How should fusion be tuned?
Using a labelled evaluation set, not intuition alone.
My Learning
Record your progress for this module.
Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.
Attempt the core lab and knowledge checks before marking the module complete.