Why this matters
- VaultCommerce's 7-day order retention means ~2 TB per broker — segment sizing affects delete performance.
- Sequential disk writes are why Kafka outperforms random-write databases for ingestion.
- Understanding segments explains offset gaps, log compaction, and disk full incidents.
- Ops teams use segment metrics to plan broker disk upgrades before Black Friday.
Partition log
Segment 0offsets 0–999
Segment 1offsets 1000–1999
Active segmentoffsets 2000+
Consumer offset 1850Points into segment 1
Segment files and indexes
Active segment receives new writes (000000000123456789.log). When segment.bytes or segment.ms is hit, the segment rolls. Sparse offset and time indexes enable O(1) offset-to-position lookup. VaultCommerce sets log.segment.bytes=1GB and log.segment.ms=604800000 (7 days max per segment).
Key points
- Segment — immutable file chunk of a partition log; active segment alone accepts writes
- Log end offset (LEO) — next offset to be written on the leader
- High watermark (HW) — offset replicated to all ISR; consumers see up to HW-1
- retention.ms — time-based segment deletion policy
- log.dirs — comma-separated data directories; VaultCommerce uses 3 NVMe mounts per broker
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 2 sections
- Retention and disk planning
- VaultCommerce rollout checklist
Test yourself
Answer these before moving on — recall is what makes it stick.