Mental model
Module 17: Use more steps only when the problem needs them- Learn routing, query decomposition, iterative retrieval and GraphRAG concepts without turning every RAG flow into an uncontrolled agent.
RAG & Knowledge Systems Engineering · Module 17 of 18
Production
Advanced & Agentic Retrieval Patterns
Learn routing, query decomposition, iterative retrieval and GraphRAG concepts without turning every RAG flow into an uncontrolled agent.
Engineering focus: Use more steps only when the problem needs them
In plain English
What problem are we solving?
Learn routing, query decomposition, iterative retrieval and GraphRAG concepts without turning every RAG flow into an uncontrolled agent.
You are not expected to know the engineering vocabulary before using the idea. Formal terminology comes after the mental model.
Running case · Evidence Desk
Improve the same system, module by module.
Carry one evidence-grounded knowledge assistant through all 18 modules. Preserve source IDs, retrieval evidence, evaluation cases, authority decisions and security assumptions so every improvement is measurable.
Mental model
Understand it before you optimize it.
Advanced retrieval is a planner with a leash. Routing, decomposition, graph search and iterative retrieval are useful only when bounded, observable and justified by the query.
01Classify query
02Choose retrieval path
03Collect evidence
04Optional next hop
05Stop on budget / sufficiency
Remember
Complexity must improve measured outcomes enough to justify added cost and failure modes.
Learning objectives
By the end of this module, you should be able to:
- Choose between single-shot, routed and iterative retrieval.
- Explain multi-hop retrieval and GraphRAG at a conceptual level.
- Bound agentic retrieval with explicit budgets and stop conditions.
Complexity should earn its place
A simple retriever is easier to test and operate. Add routing or iteration only when evaluations show a real need.
Routing chooses a knowledge path
Different query types may use different corpora, retrievers or tools.
Multi-hop questions need intermediate evidence
Some answers require first finding an entity or fact, then using that result to retrieve another source.
Agentic retrieval must be bounded
Limit steps, tools, time and cost; keep trace evidence for each decision.
Engineering depth
Engineering deep dive
- Route queries only when different retrieval paths have measurable strengths.
- Bound iterative retrieval with max steps, time/cost budgets and sufficiency checks.
- Graph-based retrieval can help entity/relationship or whole-corpus questions, but it adds indexing/query complexity.
- Keep a human-readable trace of route, sub-queries, evidence and stop reason.
Worked examples
Bounded advanced retrieval
query
↓ router
├─ basic/hybrid search
├─ entity/graph path
└─ multi-hop decomposition
step 1 evidence
step 2 evidence
step 3 STOP (budget reached or sufficient)- A stop reason is part of the trace.
- Advanced retrieval should degrade safely to a simpler path.
Diagnostic lens
If this goes wrong, where do you look first?
Symptom
An agentic retriever loops, becomes expensive or loses the evidence trail.
Likely causes
- No step budget
- Router has no confidence/fallback
- Intermediate evidence not persisted
First checks
- Count retrieval/tool steps
- Inspect route decisions
- Verify stop/sufficiency rules
Engineering principle
Bound every iterative retrieval process.
Try it · core lab · account-free
Lab 17 — Advanced & Agentic Retrieval Patterns
Goal: Route queries to two mock knowledge domains and enforce a maximum retrieval-step budget.
Before you start
- Core lab requires only Python 3 and local files unless otherwise stated.
- Running case: build the same evidence-grounded knowledge assistant across the course.
- Keep source IDs and intermediate evidence so each result is inspectable.
- Reflection: Write one sentence describing what this step proves and one sentence describing what it does not prove.
Run the baseline
python3 - <<'PY2'
def route(q): return 'network' if any(x in q.lower() for x in ['port','dns','route']) else 'general'
for q in ['Which port is open?','What is our leave policy?']: print(q,'->',route(q))
max_steps=3; print('retrieval_budget=',max_steps)
PY2Inspect the evidence
Record the intermediate source IDs, ranks, metadata or state produced by the baseline. Explain which pipeline layer this evidence belongs to.
Create a runaway loop
Simulate an iterative retriever without a stop rule, then add max_steps=3 and an explicit stop reason.
Diagnose and restore
Bound every iterative retrieval process. State the failing layer, change only that layer, then repeat the verification checks.
Write the engineering note
Record: hypothesis → evidence → change → observed result → remaining limitation. Keep this note with the running Evidence Desk project.
Verify
- Queries are routed deterministically in the toy example.
- A retrieval-step budget is explicit.
- The baseline and the deliberately broken case produce different, explainable evidence.
- You can name the pipeline layer that owns the failure and the control that repairs it.
Expected outcome
- A reproducible local result plus a short evidence note.
- A short before/after record showing the failure, diagnosis and repaired state.
Break / fix · diagnose
- Inspect the intermediate evidence before changing the model or prompt.
- If a command fails, preserve the error and classify which pipeline layer failed.
- Change one pipeline variable at a time and rerun the same test case.
- Do not accept a plausible answer as proof; verify the intermediate evidence and source identity.
Production view
Production RAG improves when every change can be traced to evidence at a specific pipeline layer. In production, retain the before/after evidence so the same failure can become a regression test.
Production checklist
What would an engineer verify before shipping this layer?
Routing justified by eval
Step/time/cost budgets enforced
Stop reason and evidence trace retained
Avoid these traps
Common beginner and engineering mistakes
- Using agents because they are fashionable.
- Allowing unbounded retrieval loops.
- Losing traceability across multi-step retrieval.
Knowledge check
Can you explain the idea and diagnose the system?
1When should you add agentic retrieval?
When simpler retrieval fails representative tasks and the added complexity measurably helps.
2What is routing?
Selecting a retriever, corpus or tool path based on the query/task.
3What is multi-hop retrieval?
Using intermediate evidence to formulate additional retrieval steps.
4What controls should bound agentic retrieval?
Step, tool, time and cost budgets plus stop conditions and tracing.
My Learning
Record your progress for this module.
Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.
Attempt the core lab and knowledge checks before marking the module complete.