Why this matters
- StreamHub's checkout path touches 6 services — without distributed tracing, a 3-second latency spike is impossible to diagnose from logs alone.
- SLOs and error budgets turn observability data into engineering priorities: burn rate alerts fire before users notice.
- OpenTelemetry (OTel) is the vendor-neutral standard in 2024–2026; instrument once, export to Prometheus, Jaeger, Grafana, or Datadog.
Observability stack components
- Instrumentation — OTel SDK in each service; auto-instrument HTTP, gRPC, DB, Kafka.
- Collector — receives OTLP data; processes, batches, and routes to backends.
- Metrics backend — Prometheus scrapes or receives remote-write; Grafana visualises.
- Logs backend — Loki, Elasticsearch, or CloudWatch Logs; correlated by trace ID.
- Traces backend — Jaeger, Tempo, or Honeycomb; distributed request timelines.
- Alerting — Alertmanager or Grafana Alerting → PagerDuty/Slack on SLO breach.
Pipeline architecture
Observability on AWS
OpenTelemetry configuration
# otel-collector-config.yaml
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
batch:
timeout: 5s
send_batch_size: 1024
memory_limiter:
check_interval: 1s
limit_mib: 512
exporters:
prometheus:
endpoint: 0.0.0.0:8889
otlp/jaeger:
endpoint: jaeger:4317
tls:
insecure: true
loki:
endpoint: http://loki:3100/loki/api/v1/push
service:
pipelines:
metrics:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [prometheus]
traces:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [otlp/jaeger]
logs:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [loki]
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 3 sections
- Application instrumentation
- SLOs and error budgets
- Structured logging with trace correlation
Test yourself
Answer these before moving on — recall is what makes it stick.