Skip to main content
PreparationZone Logo
PrepZone

Metrics, JMX, Prometheus, and Lag Alerts

Under-replicated partitions, offline replicas, and consumer lag alerts that wake the on-call engineer.

Pass checkpoint below

Why this matters

  • VaultCommerce PagerDuty fires on UnderReplicatedPartitions > 0 for 5 minutes.
  • Consumer lag SLO: inventory < 5k, analytics < 50k during peak.
  • JMX → Prometheus via kafka_exporter; Grafana dashboards per team.
  • Metrics without alerts are vanity — tie every chart to a runbook step.
Kafka brokersJMX
Prometheus
Grafana + PagerDuty
JMX metrics exported to Prometheus. Lag alerts route to on-call. Logs correlate via trace IDs.

Critical broker metrics

UnderReplicatedPartitions, OfflineReplicaCount, ActiveControllerCount (must be 1). Request handler idle percent below 20% signals overload. VaultCommerce alerts on disk above 75% and LogFlushRateAndTimeMs p99 spike.

Key points

  • Under-replicated partitions — ISR < RF; precursor to data loss risk
  • Request queue time — broker overload indicator
  • Consumer lag — offset behind log end
  • kafka_exporter — Prometheus metrics from JMX
  • Alert SLO — threshold + duration before page

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 2 sections

  • Consumer lag alerts
  • VaultCommerce rollout checklist

Test yourself

Answer these before moving on — recall is what makes it stick.