Gnu Group Learning AcademyGnu Group ↗
AI Engineering Fundamentals · Module 08 of 12

Test whether the AI actually works

Turn quality into evidence. Build evaluation cases, measure important failure slices, compare versions and prevent prompt, model or retrieval changes from silently making the application worse.

Engineering focus: Evaluation, testing and regression engineering

In plain English

What problem are we solving?

Turn quality into evidence. Build evaluation cases, measure important failure slices, compare versions and prevent prompt, model or retrieval changes from silently making the application worse.

The rest of this module introduces the formal terminology and production controls behind that idea. You are not expected to memorize the vocabulary before using it.

Running case · Atlas Assist

Carry the same support assistant forward: approved KB, source citations, explicit insufficient-evidence fallback and one bounded read-only asset lookup. Reuse prior artifacts so each module adds evidence to one production AI system.

Learning objectives

By the end of this module, you should be able to:

  • Build an evaluation dataset from real task distributions and known failure modes.
  • Choose deterministic, model-based and human evaluation methods appropriately.
  • Use baselines, slices and confidence intervals rather than one anecdotal success rate.

Evals are the unit tests of behaviour—but not only unit tests

AI systems are variable and often judged on semantic criteria. Use deterministic assertions where possible, then add rubric-based or human evaluation where correctness cannot be captured mechanically.

A representative dataset is an engineering asset

Include normal cases, edge cases, abstention cases, adversarial cases and important user segments. Avoid building only from prompts that the current system already handles.

Evaluate components and the end-to-end system

Retrieval recall, schema validity, tool success and latency can be tested separately. End-to-end task success tells whether the whole system meets the user contract.

Model graders need calibration

A model can help score relevance or style, but the grader itself has bias and variance. Calibrate against human-labelled examples, hide irrelevant metadata and use explicit rubrics.

Regression gates need thresholds and slices

Averages can hide damage to critical categories. Compare candidate versus baseline on safety, abstention, retrieval, tool use, latency and cost slices; define which metrics are blocking.

Production feedback completes the loop

Offline evals cannot represent every user. Sample failures, collect explicit feedback carefully, label new cases and add them to regression sets without leaking sensitive data.

Worked examples

Evaluation stack

deterministic checks: schema · citations · tool args · latency
rubric checks: groundedness · usefulness · policy
human review: ambiguous/high-impact quality
production: feedback + failure sampling → new eval cases
  • Use the cheapest reliable evaluator for each property.

Candidate decision

candidate wins only if:
critical safety slices do not regress
AND task-success target improves or holds
AND latency/cost stay within budget
AND sample size is sufficient to trust the conclusion
  • One aggregate score should not override a critical regression.
Try it · core lab · account-free

Lab 08 — Build an evaluation harness

Goal: Create a tiny JSONL dataset and score deterministic behavioural criteria.

Before you start

  • No paid API or cloud account is required for the core lab.
  • Core labs use Python 3 and local files only.
  • Running case: you are engineering 'Atlas Assist', an internal support assistant over a small approved knowledge base with one read-only inventory tool.
  • Carry requirements, eval cases, threat assumptions and operational evidence forward across modules instead of starting over.
STEP 1

Create evaluation cases

cat > /tmp/ai-evals.jsonl <<'EOF'
{"id":"e1","kind":"grounded","expected_status":"ANSWERED","required_source":"kb-17"}
{"id":"e2","kind":"abstain","expected_status":"INSUFFICIENT_EVIDENCE","required_source":null}
{"id":"e3","kind":"injection","expected_status":"ANSWERED","required_source":"kb-17"}
EOF
STEP 2

Score mock candidate outputs

python3 - <<'PY2'
import json
cases=[json.loads(x) for x in open('/tmp/ai-evals.jsonl')]
out={'e1':{'status':'ANSWERED','sources':['kb-17']},'e2':{'status':'INSUFFICIENT_EVIDENCE','sources':[]},'e3':{'status':'ANSWERED','sources':['kb-17']}}
passed=0
for c in cases:
 r=out[c['id']]; ok=r['status']==c['expected_status'] and (c['required_source'] is None or c['required_source'] in r['sources']); print(c['id'],ok); passed+=ok
print('score',passed,'/',len(cases)); assert passed==len(cases)
PY2
STEP 3

Define blocking slices

Mark at least security/injection and unauthorized-tool cases as non-negotiable blockers even if aggregate task success improves.

Verify

  • The dataset includes a normal, abstention and adversarial case.
  • At least one critical slice can block promotion independently of the average.

Expected outcome

  • A repeatable evaluation dataset, scorer and release decision rule.

Break / fix · diagnose

  • If all cases were written after seeing candidate outputs, create a held-out set.
  • If a model grader is used, calibrate it against human labels before trusting the score.
Production view

High-performing AI teams treat eval datasets and regression gates as versioned product assets, not as a one-time benchmark.

Local real-tool practicum · optional but recommended

Optional practicum — run the bundled eval harness

Use the downloadable lab pack's deterministic eval harness, then swap in saved local-model outputs without changing the rubric.

Prerequisites

  • Ollama installed and reachable on http://127.0.0.1:11434 for the real-tool path.

Resources created

  • Only local disposable files/processes used by the practicum.

Cost / risk

  • No cloud account required; model downloads consume disk and network bandwidth.
TOOL 1
unzip -q public/labs/ai/AI-Engineering-Lab-Pack-v0.3.19-r2.zip -d /tmp/ai-evals-pack
TOOL 2
python3 /tmp/ai-evals-pack/evals/evaluate.py /tmp/ai-evals-pack/evals/sample-results.jsonl

Verify

  • The harness reports per-case and aggregate results and exits non-zero on a blocking failure.

Cleanup

  • Remove only the disposable files created for the lab; keep model caches only if you intend to reuse them.

Warnings

  • Do not use production conversations as eval data without privacy review, minimization and appropriate retention controls.
Avoid these traps

Common mistakes

  • Judging quality from five hand-picked prompts.
  • Using one aggregate score with no critical slices.
  • Treating a model grader as ground truth.
  • Changing model, prompt and retrieval simultaneously without attribution.
Knowledge check

Can you connect the AI behaviour to a system control?

Concept and scenario checks. Reveal the answer only after you commit to a decision.

1Why are evals essential for generative systems?

Output variability and semantic quality require repeatable behavioural measurements beyond exact string assertions.

2What is a critical slice?

A subset such as security or abstention whose regression can block promotion even if the overall average improves.

3Why calibrate model graders?

They have their own bias/variance and must be compared with human-labelled examples before being trusted.

4Scenario: candidate quality rises 4% overall but prompt-injection resistance drops sharply. What should promotion do?

Block according to the critical safety slice rather than letting the average hide the regression.

My Learning

Record your progress for this module.

Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.

Not started

Attempt the core lab and knowledge checks before marking the module complete.