Skip to main content
PreparationZone Logo
PrepZone

Observability Stack

Metrics, logs, traces, SLOs and error budgets with OpenTelemetry and modern dashboards.

Pass checkpoint below

Why this matters

  • StreamHub's checkout path touches 6 services — without distributed tracing, a 3-second latency spike is impossible to diagnose from logs alone.
  • SLOs and error budgets turn observability data into engineering priorities: burn rate alerts fire before users notice.
  • OpenTelemetry (OTel) is the vendor-neutral standard in 2024–2026; instrument once, export to Prometheus, Jaeger, Grafana, or Datadog.

Observability stack components

  • Instrumentation — OTel SDK in each service; auto-instrument HTTP, gRPC, DB, Kafka.
  • Collector — receives OTLP data; processes, batches, and routes to backends.
  • Metrics backend — Prometheus scrapes or receives remote-write; Grafana visualises.
  • Logs backend — Loki, Elasticsearch, or CloudWatch Logs; correlated by trace ID.
  • Traces backend — Jaeger, Tempo, or Honeycomb; distributed request timelines.
  • Alerting — Alertmanager or Grafana Alerting → PagerDuty/Slack on SLO breach.

Pipeline architecture

Observability on AWS

OTLPalertCOMPUTE
EKS podsOTel SDK
OPS
ADOT collectorOTLP
OPS
CloudWatchmetrics · logs
OPS
X-Raytraces
OPS
Managed Grafanadashboards
EXTERNAL
PagerDutySLO alerts
OpenTelemetry from EKS → ADOT → CloudWatch + X-Ray + Grafana → PagerDuty.

OpenTelemetry configuration

Java
# otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s
    send_batch_size: 1024
  memory_limiter:
    check_interval: 1s
    limit_mib: 512

exporters:
  prometheus:
    endpoint: 0.0.0.0:8889
  otlp/jaeger:
    endpoint: jaeger:4317
    tls:
      insecure: true
  loki:
    endpoint: http://loki:3100/loki/api/v1/push

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [prometheus]
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp/jaeger]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [loki]

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 3 sections

  • Application instrumentation
  • SLOs and error budgets
  • Structured logging with trace correlation

Test yourself

Answer these before moving on — recall is what makes it stick.