Why this matters
- VaultCommerce inventory used to decrement twice when Kafka retried a consumer that crashed after processing but before commit.
- At-least-once is Kafka's default; exactly-once requires idempotent producers, transactions, and careful consumer design.
- Production incidents from misunderstood semantics cost more than any tuning knob.
- Interviewers ask you to map semantics to VaultCommerce payment and inventory flows.
At-most-once
Producer sends with acks=0 or consumer commits offset before processing. Messages may be lost but never duplicated. VaultCommerce uses at-most-once only for non-critical metrics where approximate counts suffice.
Key points
- At-most-once — may lose messages; never duplicates
- At-least-once — never loses (with proper acks); may duplicate
- Exactly-once — broker-level atomicity; app must still handle external stores
- Idempotent consumer — processing the same message twice yields the same state
- Commit order — process then commit offset; never commit before side effects are durable
At-least-once (Kafka default)
Producer retries until ack; consumer processes then commits offset. If the consumer crashes between processing and commit, the message is redelivered. VaultCommerce inventory handlers use UPDATE stock SET qty = qty - :n WHERE sku = :sku AND qty >= :n so duplicates are safe.
Exactly-once semantics (EOS)
Kafka transactions plus idempotent producers give atomic write across partitions within the broker's guarantees. Application EOS still needs idempotent side effects or transactional outbox. VaultCommerce enables EOS for payment-to-ledger pipelines in module 8.
// VaultCommerce inventory-consumer — at-least-once safe handler
@KafkaListener(topics = "vaultcommerce.orders.placed.v1", groupId = "inventory-service")
public void onOrderPlaced(ConsumerRecord<String, OrderPlaced> record) {
String eventId = record.headers().lastHeader("eventId").value();
if (processedEvents.exists(eventId)) return; // idempotency guard
inventoryService.reserve(record.value().lineItems());
processedEvents.mark(eventId); // dedup table in Postgres
// offset committed by container AFTER this method returns successfully
}
Quick recall
Everything you need if you only revisit this box.
- Kafka default is at-least-once — design consumers for duplicates.
- Commit offset only after side effects are durable.
- EOS needs broker transactions AND application idempotency.
Test yourself
Answer these before moving on — recall is what makes it stick.
- Explain at-least-once, at-most-once, and exactly-once delivery semantics—how to implement them in Kafka?
- Describe a real-world example: designing an event-driven order processing pipeline with Kafka—what are the key design decisions?
- What is idempotence for Kafka producers and how does it help with exactly-once semantics?