RAG & Knowledge Systems Engineering · Module 02 of 18

Foundation

Your First RAG System

Build a tiny retrieval-and-generation flow so the architecture becomes concrete before introducing embeddings or vector databases.

Engineering focus: Build the smallest working RAG loop

In plain English

What problem are we solving?

Build a tiny retrieval-and-generation flow so the architecture becomes concrete before introducing embeddings or vector databases.

You are not expected to know the engineering vocabulary before using the idea. Formal terminology comes after the mental model.

Running case · Evidence Desk

Improve the same system, module by module.

Carry one evidence-grounded knowledge assistant through all 18 modules. Preserve source IDs, retrieval evidence, evaluation cases, authority decisions and security assumptions so every improvement is measurable.

Python bridge RAG glossary

Mental model

Understand it before you optimize it.

A minimal RAG system is three boxes connected in order: find, package, answer. Keeping those boxes separate makes failures visible.

01retrieve(query)

02build_context(hits)

03answer(query, context)

04return answer + sources

Remember

If you cannot inspect what was retrieved, you cannot reliably debug the answer.

Learning objectives

By the end of this module, you should be able to:

Implement a minimal retrieve-then-answer pipeline.
Return the supporting source with the answer.
Add an insufficient-evidence fallback.

Start simple before adding infrastructure

A production stack may contain parsers, embeddings, vector indexes, rerankers and observability. But the learning path starts with a tiny corpus and deterministic retrieval so every moving part is visible.

Grounding contract

The answer should use supplied evidence, cite it, and abstain when evidence is insufficient.

Separation of concerns

Keep retrieval, prompt construction and generation as separate functions. That makes failures diagnosable.

Engineering depth

Engineering deep dive

Design explicit interfaces for retriever, context builder and generator.
Return structured status such as ANSWERED / INSUFFICIENT_EVIDENCE.
Preserve query, source IDs and context for reproducible debugging.
Add deterministic tests before introducing a real LLM.

Worked examples

Mental model

Module 02: Build the smallest working RAG loop

Build a tiny retrieval-and-generation flow so the architecture becomes concrete before introducing embeddings or vector databases.

Make the pipeline visible

retrieve(query) -> hits
build_context(hits) -> context + source_ids
generate(query, context) -> answer
validate(answer, source_ids) -> ANSWERED | INSUFFICIENT_EVIDENCE

Each function can be tested independently.
A trace can show exactly where a failure began.

Diagnostic lens

If this goes wrong, where do you look first?

Symptom

A wrong answer appears and the team cannot tell whether retrieval or generation failed.

Likely causes

Retrieval and generation are coupled
Intermediate context is not logged
No abstention state exists

First checks

Print retrieved records
Print the exact context packet
Run the same query with generation disabled

Engineering principle

Separate pipeline stages and make their inputs/outputs inspectable.

Try it · core lab · account-free

Lab 02 — Your First RAG System

Goal: Build a minimal Python retriever over three text records and return source IDs with the answer context.

Before you start

Core lab requires only Python 3 and local files unless otherwise stated.
Running case: build the same evidence-grounded knowledge assistant across the course.
Keep source IDs and intermediate evidence so each result is inspectable.
Reflection: Write one sentence describing what this step proves and one sentence describing what it does not prove.

STEP 1

Run the baseline

python3 - <<'PY2'
docs={'net.md':'The service listens on port 8080.','auth.md':'VPN reset requires MFA.','hr.md':'Payroll closes Friday.'}
q='Which port does the service use?'
hits=[(k,v) for k,v in docs.items() if 'port' in v.lower() or '8080' in v]
assert hits
print({'context':hits[0][1],'source':hits[0][0]})
PY2

STEP 2

Inspect the evidence

Record the intermediate source IDs, ranks, metadata or state produced by the baseline. Explain which pipeline layer this evidence belongs to.

STEP 3

Test abstention

Change the query to an unrelated question. Confirm retrieval returns no evidence and define the structured INSUFFICIENT_EVIDENCE result.

STEP 4

Diagnose and restore

Separate pipeline stages and make their inputs/outputs inspectable. State the failing layer, change only that layer, then repeat the verification checks.

STEP 5

Write the engineering note

Record: hypothesis → evidence → change → observed result → remaining limitation. Keep this note with the running Evidence Desk project.

Verify

The result contains both evidence and source identity.
An unrelated question returns no hit and should trigger abstention.
The baseline and the deliberately broken case produce different, explainable evidence.
You can name the pipeline layer that owns the failure and the control that repairs it.

Expected outcome

A reproducible local result plus a short evidence note.
A short before/after record showing the failure, diagnosis and repaired state.

Break / fix · diagnose

Inspect the intermediate evidence before changing the model or prompt.
If a command fails, preserve the error and classify which pipeline layer failed.
Change one pipeline variable at a time and rerun the same test case.
Do not accept a plausible answer as proof; verify the intermediate evidence and source identity.

Production view

Production RAG improves when every change can be traced to evidence at a specific pipeline layer. In production, retain the before/after evidence so the same failure can become a regression test.

Production checklist

What would an engineer verify before shipping this layer?

Gate 1

Stage interfaces are testable

Gate 2

No-evidence status is explicit

Gate 3

Source IDs survive to output

Avoid these traps

Common beginner and engineering mistakes

Hiding retrieval inside one giant function.
Returning an answer without a source.
Forcing an answer when no source matches.

Knowledge check

Can you explain the idea and diagnose the system?

1Why keep retrieval separate from generation?

So retrieval failures can be distinguished from generation failures.

2What should happen when no evidence is found?

Return an explicit insufficient-evidence state rather than fabricate an answer.

3Why return source IDs?

They provide provenance for debugging, citations and auditability.

4What is the goal of the first RAG system?

Understand the control flow, not maximize retrieval sophistication.

My Learning

Record your progress for this module.

Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.

Not started

Core lab attemptedRecord this after meaningfully completing or attempting the module's core lab.Knowledge checks attemptedThis is marked automatically after you open every answer in the module; you can also record it manually.

Attempt the core lab and knowledge checks before marking the module complete.