Mental model
Module 11: Make answers show their evidence- Design outputs that distinguish supported answers from unsupported guesses and make citations useful for verification.
RAG & Knowledge Systems Engineering · Module 11 of 18
Engineer
Grounding, Citations & Abstention
Design outputs that distinguish supported answers from unsupported guesses and make citations useful for verification.
Engineering focus: Make answers show their evidence
In plain English
What problem are we solving?
Design outputs that distinguish supported answers from unsupported guesses and make citations useful for verification.
You are not expected to know the engineering vocabulary before using the idea. Formal terminology comes after the mental model.
Running case · Evidence Desk
Improve the same system, module by module.
Carry one evidence-grounded knowledge assistant through all 18 modules. Preserve source IDs, retrieval evidence, evaluation cases, authority decisions and security assumptions so every improvement is measurable.
Mental model
Understand it before you optimize it.
A grounded answer is a set of claims connected to evidence. If the evidence cannot support a claim, the system should weaken, qualify or refuse that claim.
01Evidence
02Claims
03Claim→source mapping
04Citation
05Answer or abstain
Remember
Citations are useful only when they actually support the claim they are attached to.
Learning objectives
By the end of this module, you should be able to:
- Define groundedness in operational terms.
- Require claim-to-source support.
- Implement an explicit abstention state.
Grounding is a relationship between claims and evidence
A response is grounded when its important factual claims are supported by the evidence actually supplied.
Citations should be verifiable
A citation must point to a source/passage that supports the claim, not merely a document that discusses the same topic.
Abstention is a feature
When evidence is missing, ambiguous or conflicting, saying 'insufficient evidence' can be the correct production behaviour.
Separate answer quality from evidence quality
A stylistically excellent answer can still be ungrounded; a strong evidence set can still be misused by generation.
Engineering depth
Engineering deep dive
- Separate factual support, answer relevance and stylistic fluency.
- Represent abstention as a valid product outcome with user-facing next steps.
- Use claim-level citation checks for high-stakes or audit-sensitive answers.
- Do not infer confidence from model tone.
Worked examples
Claim-to-source map
Claim 1: "API uses TCP 8080"
-> network.md#api [SUPPORTED]
Claim 2: "API is public internet-facing"
-> no supporting chunk [UNSUPPORTED]
Final answer keeps Claim 1 and omits/qualifies Claim 2.- A citation must support the exact claim.
- Unsupported claims become visible before release.
Diagnostic lens
If this goes wrong, where do you look first?
Symptom
The answer includes a citation, but the cited passage does not support the claim.
Likely causes
- Citation generated independently of claims
- Wrong chunk ID mapping
- Generator added unsupported detail
First checks
- Split answer into claims
- Map each claim to retrieved chunk
- Run unsupported-claim negative tests
Engineering principle
Verify claim-to-evidence support, not citation presence.
Try it · core lab · account-free
Lab 11 — Grounding, Citations & Abstention
Goal: Validate a structured answer that requires source IDs or an INSUFFICIENT_EVIDENCE status.
Before you start
- Core lab requires only Python 3 and local files unless otherwise stated.
- Running case: build the same evidence-grounded knowledge assistant across the course.
- Keep source IDs and intermediate evidence so each result is inspectable.
- Reflection: Write one sentence describing what this step proves and one sentence describing what it does not prove.
Run the baseline
python3 - <<'PY2'
result={'status':'ANSWERED','answer':'The API uses port 8080.','source_ids':['api.md#network']}
assert result['status'] in {'ANSWERED','INSUFFICIENT_EVIDENCE'}
assert result['status']!='ANSWERED' or result['source_ids']
print('PASS')
PY2Inspect the evidence
Record the intermediate source IDs, ranks, metadata or state produced by the baseline. Explain which pipeline layer this evidence belongs to.
Inject an unsupported claim
Add one statement that is not present in any retrieved source. Mark it unsupported and remove or qualify it before final output.
Diagnose and restore
Verify claim-to-evidence support, not citation presence. State the failing layer, change only that layer, then repeat the verification checks.
Write the engineering note
Record: hypothesis → evidence → change → observed result → remaining limitation. Keep this note with the running Evidence Desk project.
Verify
- Answered responses require citations.
- No-evidence cases can be represented without fabricating content.
- The baseline and the deliberately broken case produce different, explainable evidence.
- You can name the pipeline layer that owns the failure and the control that repairs it.
Expected outcome
- A reproducible local result plus a short evidence note.
- A short before/after record showing the failure, diagnosis and repaired state.
Break / fix · diagnose
- Inspect the intermediate evidence before changing the model or prompt.
- If a command fails, preserve the error and classify which pipeline layer failed.
- Change one pipeline variable at a time and rerun the same test case.
- Do not accept a plausible answer as proof; verify the intermediate evidence and source identity.
Production view
Production RAG improves when every change can be traced to evidence at a specific pipeline layer. In production, retain the before/after evidence so the same failure can become a regression test.
Production checklist
What would an engineer verify before shipping this layer?
Claim support checked
Abstention path tested
Citations resolve to exact evidence
Avoid these traps
Common beginner and engineering mistakes
- Citing a source that does not support the specific claim.
- Forcing the model to always answer.
- Equating confidence wording with evidence strength.
Knowledge check
Can you explain the idea and diagnose the system?
1What is groundedness?
The degree to which generated claims are supported by supplied evidence.
2What makes a citation useful?
It points to evidence that actually supports the associated claim.
3Why is abstention important?
It prevents unsupported answers when evidence is insufficient.
4Can a fluent answer be ungrounded?
Yes. Fluency and evidential support are different properties.
My Learning
Record your progress for this module.
Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.
Attempt the core lab and knowledge checks before marking the module complete.