Skip to main content
PreparationZone Logo
PrepZone

Batching, linger.ms, and Compression

Trade a few milliseconds of latency for 10x throughput — batching and compression on the VaultCommerce producer.

Pass checkpoint below

Why this matters

  • VaultCommerce cut producer CPU network utilization 60% with compression.type=lz4 and linger.ms=5.
  • Batching is why Kafka achieves millions of msgs/sec — not single-record sends.
  • batch.size and buffer.memory bound memory per producer instance.
  • Tuning batching is the first throughput optimization after correctness settings.
ApplicationVaultCommerce OrderService
Serializer
Record batch
Broker leader
Serializer → partitioner → record batch → broker leader. Callback fires with RecordMetadata or error.

linger.ms and batch.size

Producer waits up to linger.ms to fill a batch up to batch.size bytes before sending. VaultCommerce order events (~400 byte Avro) batch ~50 records per partition per 5ms window. Lower linger for latency-sensitive payment capture topic (linger.ms=0).

Key points

  • batch.size — target batch bytes per partition before flush
  • linger.ms — max wait to fill a batch
  • buffer.memory — total memory for unsent batches (default 32MB)
  • compression.type — lz4, zstd, snappy, gzip, or none
  • RecordAccumulator — internal queue organizing batches by partition

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 2 sections

  • Compression codecs
  • VaultCommerce rollout checklist

Test yourself

Answer these before moving on — recall is what makes it stick.