Why this matters
- VaultCommerce still uses RabbitMQ for 200 admin alerts/day while Kafka handles 2M order events/day.
- Replay capability saved the analytics rebuild — impossible on SQS without a separate archive.
- Operational cost: Kafka cluster vs managed SQS — trade headcount for dollars at smaller scale.
- System design interviews pair this with the RabbitMQ/Kafka patterns article.
When VaultCommerce chose Kafka
Requirements: 7-day retention, 5 independent consumer teams, peak 50k events/sec, schema evolution with Avro. RabbitMQ hit head-of-line blocking at 8k/s with durable queues. SQS lacked ordering without FIFO shards. Kafka's partition model matched.
Key points
- Replay — new consumer group reads history; queues delete on ack
- Fan-out — multiple consumer groups on one Kafka topic vs multiple Rabbit queues bound to exchange
- Ordering — Kafka per-partition; Rabbit per-queue; SQS FIFO per message group
- Ops burden — self-managed Kafka vs managed SQS/Confluent Cloud
- Payload model — Kafka log retention vs queue message TTL
When queues still win
Low volume, complex routing (topic exchange with per-tenant bindings), per-message TTL, and priority queues — RabbitMQ is simpler. VaultCommerce's refund approval workflow uses RabbitMQ with 3 consumers and DLQ in 50 lines of config.
VaultCommerce rollout checklist
Before promoting changes that touch the VaultCommerce order and payment event backbone, run the staging KRaft cluster (Kafka 3.7+, Schema Registry 7.x) through a 10k events/min soak test. Compare producer request latency p99 and consumer lag per group against the pre-deploy baseline. Kafka for high-throughput logs with replay. Document the change in the internal topic registry, attach Grafana screenshots to the change ticket, and keep an engineer on lag dashboards for 30 minutes after production rollout — roll back the service release before altering broker-level settings if lag or under-replicated partitions spike.
// VaultCommerce — same OrderPlaced event, different middleware choices
// Kafka: retained log, multiple independent consumer groups
producer.send(new ProducerRecord<>("vaultcommerce.orders.placed.v1", orderId, avroBytes));
// RabbitMQ (admin alerts only): routed, deleted after ack
rabbitTemplate.convertAndSend("vaultcommerce.admin", "inventory.low-stock", alertJson);
Quick recall
Everything you need if you only revisit this box.
- Kafka for high-throughput logs with replay.
- Queues for task distribution and simple routing.
- VaultCommerce uses both — right tool per workload.
System Design trackRabbitMQ and Kafka patterns — full system design comparison
Test yourself
Answer these before moving on — recall is what makes it stick.