Learning objectives
By the end of this module, you should be able to:
- Choose visualizations based on the question.
- Design a service overview around traffic, errors, latency and saturation.
- Use dashboard variables and drill-downs without hiding scope.
Dashboards should support decisions
Every panel should answer an operational question: Is user experience degrading? Which route or region? Is a resource saturated? What changed?
Overview first, drill down second
A good service dashboard exposes health and scope, then provides route/instance/dependency drill-downs. Avoid forcing users to scan dozens of equally weighted panels.
Visualization can distort
Axis ranges, units, aggregation and time windows change perception. A dashboard should expose units, scope and meaningful thresholds rather than rely on visual drama.
Worked example
See the investigation idea in practice.
Dashboard hierarchy
ROW 1: traffic | error ratio | p95 latency | saturation
ROW 2: by route / region
ROW 3: dependency health + deploy annotations
LINKS: logs | traces | runbook
- Read the evidence before deciding what it means.
- Prefer a hypothesis that explains multiple independent signals.
Core lab · account-free
Lab 05 — Review a dashboard specification
Goal: Detect panels that are redundant, unscoped or likely to hide an incident.
Before you start
- No cloud account required.
- Core lab uses bash/Python 3 and local files only.
STEP 1Create dashboard spec
cat > /tmp/dashboard.txt <<'EOF'
requests_per_sec by route
error_ratio by route
p95_latency_ms by route
app_cpu_pct by instance
db_pool_pct
deploy_annotation
EOF
cat /tmp/dashboard.txt
STEP 2Map each panel to a question
For every line, write the operational question it should answer.
STEP 3Add drill-down links
Add one logs link and one traces link that carry service/time context.
Verify
- Each panel has a question and unit/scope.
- Dashboard includes both user symptom and dependency/resource evidence.
Expected outcome
- A compact service overview with an investigation path rather than a wall of charts.
Diagnose / troubleshoot
- If two panels answer the same question, remove or differentiate one.
- If a panel says “high” without units/baseline, make the threshold and unit explicit.
Real-world connectionDashboards are most effective when designed around incident decisions and team runbooks.
Optional real-tool lab
Optional tool lab — inspect provisioned Grafana
Verify the local Prometheus data source and Grafana health.
Resources created
- Local containers/files only; no cloud resources.
Cost / risk
- Expected cloud cost: none. Local CPU, memory and disk usage only.
Prerequisites
- Docker Engine with Compose plugin for the optional shared stack.
- Core course completion does not require Docker or external credentials.
TOOL 1cd labs/observability-stack
TOOL 2docker compose up -d
TOOL 3curl -fsS -u admin:academy-lab http://127.0.0.1:3001/api/datasources
Verify
- A Prometheus data source is provisioned and points at the local Prometheus service.
Cleanup
- Run `docker compose down -v` from `labs/observability-stack`.
Warnings
- Optional only: not required by the account-free validator.
- The bundled local stack is for training, not a production observability architecture.
Avoid these traps
Common mistakes
- Maximizing panel count.
- Using pie charts for time-series investigation.
- Showing averages when tail latency matters.
Knowledge check
Can you reason from the signal?
Three concept checks plus one scenario check. Open each answer only after deciding first.
1What should determine a dashboard panel?
A concrete operational question or decision.
2Why show p95/p99 latency instead of only average?
Tail latency can reveal poor user experience hidden by a healthy average.
3What is a useful dashboard drill-down?
A context-preserving link from a metric symptom to relevant logs or traces.
4Scenario: a dashboard has 40 panels but responders cannot tell which route is failing. What is the design problem?
The dashboard lacks hierarchy/scope and should preserve actionable dimensions such as route before adding more panels.