Gnu Group Learning AcademyGnu Group ↗
AI Engineering Fundamentals · Module 06 of 12

Let AI answer from your documents

Build a retrieval-augmented generation (RAG) pipeline that finds relevant evidence, gives it to the model, cites sources and abstains when the available evidence is not good enough.

Engineering focus: RAG: grounding, citations and failure modes

In plain English

What problem are we solving?

Build a retrieval-augmented generation (RAG) pipeline that finds relevant evidence, gives it to the model, cites sources and abstains when the available evidence is not good enough.

The rest of this module introduces the formal terminology and production controls behind that idea. You are not expected to memorize the vocabulary before using it.

Running case · Atlas Assist

Carry the same support assistant forward: approved KB, source citations, explicit insufficient-evidence fallback and one bounded read-only asset lookup. Reuse prior artifacts so each module adds evidence to one production AI system.

Learning objectives

By the end of this module, you should be able to:

  • Design a RAG pipeline with explicit retrieval, grounding and fallback stages.
  • Require claim-to-source attribution and define what counts as sufficient evidence.
  • Diagnose retrieval, context and generation failure modes separately.

RAG changes the evidence available to the model

Retrieval-augmented generation supplies selected external evidence at inference time. It does not make the model intrinsically truthful, and it can introduce stale, irrelevant, poisoned or unauthorized context.

Grounding is a behavioural requirement

A grounded answer should be supported by the supplied evidence for the claims it makes. Citation presence alone is weak: a citation can be irrelevant or fail to support the nearby claim.

Abstention is a feature

When retrieval returns no sufficient evidence, a trustworthy assistant should follow the fallback contract. Forcing an answer turns missing evidence into hallucination risk.

Context assembly needs policy

Deduplicate near-identical chunks, preserve source IDs, prioritize authoritative/fresh sources, apply access filters before model generation and reserve context budget for the actual answer.

RAG evaluation has layers

Track retrieval recall/precision, context relevance, answer groundedness, citation correctness and task usefulness. A single aggregate score can hide the failing stage.

Worked examples

Grounded RAG

query → authorization/filter → retrieval → rerank → context builder
→ model → schema validator → citation/grounding check → answer or abstain
  • Evidence policy exists before and after generation.

Failure taxonomy

MISS: evidence not retrieved
NOISE: irrelevant evidence dominates
CONFLICT: sources disagree
STALE: old version retrieved
INJECTION: retrieved text attacks instructions
UNGROUNDED: answer adds unsupported claim
  • Each failure suggests a different fix.
Try it · core lab · account-free

Lab 06 — Build a grounded answer assembler

Goal: Use the KB and retrieval cases to create answer records that must carry source IDs and abstain when evidence is missing.

Before you start

  • No paid API or cloud account is required for the core lab.
  • Core labs use Python 3 and local files only.
  • Running case: you are engineering 'Atlas Assist', an internal support assistant over a small approved knowledge base with one read-only inventory tool.
  • Carry requirements, eval cases, threat assumptions and operational evidence forward across modules instead of starting over.
STEP 1

Create grounded cases

python3 - <<'PY2'
cases=[
 {'q':'When is maintenance?','retrieved':['kb-22'],'answer':'Sunday at 02:00 UTC.','sources':['kb-22'],'status':'ANSWERED'},
 {'q':'What is the VPN password?','retrieved':[],'answer':'','sources':[],'status':'INSUFFICIENT_EVIDENCE'}
]
for c in cases:
 if c['status']=='ANSWERED': assert c['sources'] and set(c['sources'])<=set(c['retrieved'])
 else: assert not c['answer'] and not c['sources']
print('PASS: answer/fallback contract',len(cases))
PY2
STEP 2

Add a conflicting-source case

Create two KB snippets with different maintenance windows and specify which metadata (effective date/authority) the retriever must use to resolve or expose the conflict.

STEP 3

Define citation correctness

For each answer claim, record the source passage that supports it. Do not give credit merely because a source ID appears somewhere in the response.

Verify

  • Unsupported questions produce the explicit fallback state.
  • Answered cases cite retrieved evidence that actually supports the claim.

Expected outcome

  • A layered RAG test set with answered, missing-evidence and conflicting-source cases.

Break / fix · diagnose

  • If every query gets an answer, test the fallback path intentionally.
  • If citations are present but unrelated, add claim-level citation checks.
Production view

Enterprise RAG quality depends on evidence selection and attribution, not only model eloquence.

Local real-tool practicum · optional but recommended

Optional practicum — local RAG request

Use the lab pack to retrieve a small approved context and send only that context to a local chat model.

Prerequisites

  • Ollama installed and reachable on http://127.0.0.1:11434 for the real-tool path.

Resources created

  • Only local disposable files/processes used by the practicum.

Cost / risk

  • No cloud account required; model downloads consume disk and network bandwidth.
TOOL 1
unzip -q public/labs/ai/AI-Engineering-Lab-Pack-v0.3.19-r2.zip -d /tmp/ai-lab-pack
TOOL 2
python3 /tmp/ai-lab-pack/scripts/simple_retriever.py 'password reset'
TOOL 3
MODEL=${MODEL:-qwen3:8b}; python3 /tmp/ai-lab-pack/scripts/build_grounded_prompt.py 'password reset' > /tmp/grounded-prompt.txt; head -80 /tmp/grounded-prompt.txt

Verify

  • The prompt contains explicit source IDs and only the selected local KB evidence.

Cleanup

  • Remove only the disposable files created for the lab; keep model caches only if you intend to reuse them.

Warnings

  • Do not index production documents until document authorization, retention and injection controls are designed.
Avoid these traps

Common mistakes

  • Calling any answer with a citation grounded.
  • Forcing the model to answer when retrieval is empty.
  • Applying access filtering after generation.
  • Collapsing retrieval and generation failures into one generic 'hallucination' label.
Knowledge check

Can you connect the AI behaviour to a system control?

Concept and scenario checks. Reveal the answer only after you commit to a decision.

1What does RAG fundamentally change?

It supplies selected external evidence at inference time; it does not guarantee truthfulness.

2What is citation correctness?

The cited source actually supports the associated claim, not merely that a citation exists.

3Where should document access filtering occur?

Before unauthorized content reaches model context, with policy enforced by trusted application logic.

4Scenario: retrieval returns nothing relevant. What is the correct behaviour?

Use the explicit insufficient-evidence fallback or escalation path rather than fabricate an answer.

My Learning

Record your progress for this module.

Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.

Not started

Attempt the core lab and knowledge checks before marking the module complete.