Learning objectives
By the end of this module, you should be able to:
- Design an end-to-end reliability contract for a production-shaped service.
- Validate that monitoring, alerting, response and recovery all reference the same user objective.
- Defend reliability decisions with evidence instead of tool screenshots.
One objective should connect the system
The capstone is relationship-aware. The user journey, SLI, SLO, burn alert, incident severity, recovery check and postmortem must describe the same reliability concern.
A disconnected dashboard or generic runbook is not enough.
Evidence must survive disagreement
A reviewer should be able to inspect the event population, objective, observed failures, alert decision, incident timeline, rollback evidence and follow-up actions without relying on the learner’s narrative alone.
Make assumptions explicit and keep calculations reproducible.
Operational readiness includes humans and capacity
The final design needs ownership, escalation, safe access, capacity headroom, rollback, recovery objectives and corrective-action tracking. Reliability is a socio-technical system, not a monitoring feature.
The capstone is formative evidence, not a hidden certificate gate
The relationship-aware validator checks structural consistency, but it cannot judge whether the chosen SLO, severity decision or resilience trade-off is appropriate. Use the rubric for self-review, peer review or instructor review.
For this Academy release, certificate eligibility is determined by the timed assessment and payment/issuance workflow; the capstone is recommended learning evidence and is not an additional automated pass/fail requirement.
Worked example
Capstone evidence chain
Journey → SLI → SLO/error budget → burn alert → on-call → incident → mitigation → SLI recovery → postmortem → reliability backlog
- Every arrow must be supported by a concrete relationship.
- The validator intentionally rejects inconsistent SLO, ownership or recovery evidence.
Core lab · account-free
Lab 12 — Validate a reliability evidence bundle
Goal: Run the supplied relationship-aware validator against a production-shaped reliability record.
Before you start
- No cloud account required.
- Python 3 is required. You may use the release source tree or download the public SRE Lab Pack from the course landing page.
- Running case study: use the same checkout journey throughout the course unless the lab explicitly says otherwise.
STEP 1Get the capstone assets
if [ -f labs/sre-capstone/reliability.example.json ]; then
cp labs/sre-capstone/reliability.example.json /tmp/reliability.json
cp labs/sre-capstone/validate_reliability.py /tmp/validate_reliability.py
else
curl -fsS https://learn.gnugroup.org/academy/labs/sre/reliability.example.json -o /tmp/reliability.json
curl -fsS https://learn.gnugroup.org/academy/labs/sre/validate_reliability.py -o /tmp/validate_reliability.py
fi
STEP 2Validate the relationships
python3 /tmp/validate_reliability.py /tmp/reliability.json
STEP 3Break one relationship deliberately
python3 - <<'PY2'
import json
p='/tmp/reliability.json'
x=json.load(open(p))
x['incident']['recovery_sli']='wrong-sli'
json.dump(x,open('/tmp/reliability-bad.json','w'),indent=2)
PY2
python3 /tmp/validate_reliability.py /tmp/reliability-bad.json || echo 'expected: validator rejected inconsistent recovery evidence'
STEP 4Defend the design in writing
Write a 150–250 word defense covering: why the SLI represents the user journey; why the SLO/error-budget policy is appropriate; why the alert deserves its urgency; and which rollback/recovery evidence would change your decision. The validator cannot grade this reasoning—use the rubric for self/peer/instructor review.
Verify
- The supplied evidence bundle passes.
- Changing the incident recovery SLI to an unrelated indicator causes validation to fail.
- The learner can defend the reliability choices in prose; the validator alone is not treated as sufficient evidence.
Expected outcome
- A coherent reliability contract plus a short written defense suitable for rubric-based self, peer or instructor review.
Diagnose / troubleshoot
- If a field exists only to satisfy the validator, explain the operational decision it represents.
- If rollback or escalation has no owner, the system is not operationally complete.
Real-world connectionThe capstone demonstrates that reliability is a connected operating system of objectives, telemetry, decisions, responders and learning—not a collection of isolated practices.
Real-tool practicum · optional but recommended
Optional practicum — run an end-to-end local reliability drill
Start the demo service and Prometheus, inject failure, inspect burn, recover and save the evidence.
Prerequisites
- Docker/Compose and Python 3.
- SRE Lab Pack or release source tree.
Resources created
- Local demo service, Prometheus evidence and a capstone reliability bundle.
Cost / risk
TOOL 1./ops/setup-sre-local-lab.sh
TOOL 2for i in $(seq 1 20); do curl -s -o /dev/null http://127.0.0.1:8000/fail || true; done
TOOL 3curl -fsS http://127.0.0.1:9090/api/v1/query --get --data-urlencode 'query=sre_demo:burn_rate:1m' > /tmp/sre-burn.json
TOOL 4for i in $(seq 1 50); do curl -s -o /dev/null http://127.0.0.1:8000/ok; done
TOOL 5cp labs/sre-capstone/reliability.example.json /tmp/reliability.json && python3 labs/sre-capstone/validate_reliability.py /tmp/reliability.json
Verify
- The learner can show failure evidence, recovery evidence and a coherent capstone contract.
Cleanup
- ./ops/cleanup-sre-local-lab.sh
Warnings
- Run failure injection only in the disposable lab.
- The validator checks relationships; the written defense and rubric are still required for meaningful review.
Capstone rubric
Score connected reliability evidence—not tool polish.
User journey + SLI contract · 15%
Critical user journey, valid/good event definitions, measurement location and limitations.
SLO + error-budget policy · 15%
Target/window, exact budget calculation, stakeholders and policy actions.
Monitoring + alerting · 15%
Golden-signal context, burn-rate alert, urgency/actionability and low-traffic considerations.
On-call + incident response · 15%
Ownership, escalation, severity, roles, timeline, mitigation and communication.
Capacity + resilience · 15%
Headroom, overload controls, timeout/retry policy, graceful degradation and safe-change rollback.
Troubleshooting + runbook · 10%
Symptom-first hypotheses, discriminating checks, safe actions, verification and escalation.
Postmortem + improvement · 15%
Blameless causal analysis, accountable actions, toil/reliability roadmap and verification metrics.
Avoid these traps
Common mistakes
- Showing tool screenshots without a user reliability objective.
- Using one SLI for the SLO and a different unrelated signal to declare incident recovery.
- Leaving ownership, escalation or rollback implicit.
Knowledge check
Can you connect the reliability objective to an engineering decision?
Three concept checks plus one scenario check.
1What makes the SRE capstone relationship-aware?
The user journey, SLI, SLO, alerts, incident recovery and learning evidence must refer to a coherent reliability objective.
2Why is a dashboard screenshot insufficient capstone evidence?
It does not prove the event population, objective, alert decision, response ownership, recovery or corrective-action relationships.
3What operational evidence should accompany a reliability objective?
At minimum ownership, alert/response policy, escalation, mitigation/rollback, capacity/recovery context and measurable follow-up.
4Scenario: the incident is declared recovered because CPU normalized, but the checkout SLI is still below objective. Is recovery proven?
No. Recovery must be verified against the affected user-facing service indicator or another explicitly justified recovery criterion.