Why this matters
- VaultCommerce pages on-call when inventory lag exceeds 10k for 5 minutes during sales.
- Scaling past partition count does not reduce lag — add partitions (carefully) or optimize handlers.
- Lag growth rate matters more than absolute value for capacity planning.
- Backpressure upstream prevents unbounded lag from drowning downstream Postgres.
Log endoffset 10000
Lag gap500 messages
Consumer offsetoffset 9500
Measuring lag
kafka-consumer-groups.sh --describe shows CURRENT-OFFSET vs LOG-END-OFFSET per partition. Burrow or Kafka lag exporter pushes to Prometheus. VaultCommerce dashboards aggregate sum(lag) per service with partition breakdown for skew detection.
Key points
- Consumer lag — LOG-END-OFFSET minus CURRENT-OFFSET per partition
- Lag spike — deploy bug, slow DB, or traffic surge
- Partition skew — one partition lagging while others are fine (hot key)
- HPA ceiling — min(replicas, partition count) for Kafka consumers
- Backpressure — slow producers or shed load when consumers saturated
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 2 sections
- Scaling and backpressure
- VaultCommerce rollout checklist
Test yourself
Answer these before moving on — recall is what makes it stick.