Gnu Group Learning AcademyGnu Group ↗
Observability Fundamentals · Module 07 of 12

OpenTelemetry and the Collector

Understand vendor-neutral telemetry APIs/SDKs, OTLP, semantic conventions and the Collector pipeline used to receive, process and export signals.

Learning objectives

By the end of this module, you should be able to:

  • Explain instrumentation versus collection/export.
  • Describe receiver → processor → exporter pipelines.
  • Use semantic conventions to keep telemetry meaning consistent.

OpenTelemetry separates instrumentation from backend choice

Applications can emit telemetry through OpenTelemetry APIs/SDKs and OTLP without hard-coding one analysis backend into business logic.

Collector pipelines are explicit

A Collector receives telemetry, can batch/filter/enrich it, and exports it to one or more destinations. Pipelines are configured independently for traces, metrics and logs.

Semantic conventions improve portability

Consistent attribute names for services, HTTP, databases and messaging reduce translation work and make cross-service investigation more coherent.

Worked example

See the investigation idea in practice.

Collector pipeline

receivers:
  otlp:
    protocols:
      http:
processors:
  batch:
exporters:
  debug:
service:
  pipelines:
    traces: {receivers: [otlp], processors: [batch], exporters: [debug]}
  • Read the evidence before deciding what it means.
  • Prefer a hypothesis that explains multiple independent signals.
Core lab · account-free

Lab 07 — Validate a telemetry pipeline model

Goal: Check that every signal pipeline has a receiver, processor and exporter.

Before you start

  • No cloud account required.
  • Core lab uses bash/Python 3 and local files only.
STEP 1

Create pipeline JSON

cat > /tmp/pipeline.json <<'EOF'
{"traces":{"receivers":["otlp"],"processors":["batch"],"exporters":["debug"]},"logs":{"receivers":["otlp"],"processors":["batch"],"exporters":["debug"]}}
EOF
STEP 2

Validate shape

python3 - <<'PY'
import json
p=json.load(open('/tmp/pipeline.json'))
for name,cfg in p.items():
 assert cfg['receivers'] and cfg['processors'] and cfg['exporters']
 print(name,'pipeline OK')
PY
STEP 3

Reason about failure

Explain what evidence disappears if the receiver is reachable but the exporter cannot deliver.

Verify

  • Trace and log pipelines pass structural validation.
  • You can name where to inspect queue/export failures.

Expected outcome

  • A clear separation between application instrumentation and backend/export configuration.

Diagnose / troubleshoot

  • If the Collector starts but data never arrives, inspect receiver protocol/port and application endpoint.
  • If data arrives but is absent downstream, inspect processors, exporter credentials, queue/retry and destination health.
Real-world connection

Collector deployment creates a controllable telemetry boundary between applications and observability backends.

Optional real-tool lab

Optional tool lab — send OTLP to Collector

Post a sample OTLP/HTTP trace to the local Collector and confirm ingestion.

Resources created

  • Local containers/files only; no cloud resources.

Cost / risk

  • Expected cloud cost: none. Local CPU, memory and disk usage only.

Prerequisites

  • Docker Engine with Compose plugin for the optional shared stack.
  • Core course completion does not require Docker or external credentials.
TOOL 1
cd labs/observability-stack
TOOL 2
docker compose up -d
TOOL 3
./send-otlp-trace.sh
TOOL 4
docker compose logs otel-collector --tail=100

Verify

  • Collector debug output contains the sample service/span attributes.

Cleanup

  • Run `docker compose down -v` from `labs/observability-stack`.

Warnings

  • Optional only: not required by the account-free validator.
  • The bundled local stack is for training, not a production observability architecture.
Avoid these traps

Common mistakes

  • Assuming OpenTelemetry itself is a storage backend.
  • Changing attribute names independently in every service.
  • Treating Collector health as proof that downstream export succeeds.
Knowledge check

Can you reason from the signal?

Three concept checks plus one scenario check. Open each answer only after deciding first.

1What is OTLP?

The OpenTelemetry Protocol used to transport telemetry between SDKs, Collectors and compatible systems.

2What does a Collector processor do?

Transforms, batches, filters or enriches telemetry between receiving and exporting.

3Why use semantic conventions?

To give common telemetry attributes consistent names and meaning across services.

4Scenario: Collector receives spans but backend is empty. Which pipeline stage should you inspect next?

Exporter delivery/configuration and downstream health, including retries/queues.

My Learning

Record your progress for this module.

Progress is saved in this browser immediately. When you sign in to My Learning, it can also sync across devices.

Not started

Attempt the core lab and knowledge checks before marking the module complete.