Idempotent Producer and Sequence Numbers

Sequence numbers and PID

On first send to a partition, broker assigns a producer ID (PID). Each message gets a monotonic sequence per PID-partition pair. If broker sees duplicate sequence, it acks without appending. VaultCommerce enables idempotence on every production producer.

Limits and scope

Idempotence covers producer retries within delivery.timeout.ms — not application-level duplicate publishes from two different producer instances with the same payload. Consumer idempotency keys still required for end-to-end safety.

VaultCommerce rollout checklist

Before promoting changes that touch vaultcommerce.orders.placed.v1 and vaultcommerce.payments.captured.v1, run the staging KRaft cluster (Kafka 3.7+, Schema Registry 7.x) through a 10k events/min soak test. Compare producer request latency p99 and consumer lag per group against the pre-deploy baseline. enable.idempotence=true prevents log duplicates on retry. Document the change in the internal topic registry, attach Grafana screenshots to the change ticket, and keep an engineer on lag dashboards for 30 minutes after production rollout — roll back the service release before altering broker-level settings if lag or under-replicated partitions spike.

// VaultCommerce — idempotent producer (required before transactions)
props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
// Implicit: ACKS_CONFIG=all, RETRIES_CONFIG=Integer.MAX_VALUE,
//          MAX_IN_FLIGHT_REQUESTS_PER_CONNECTION=5

try (KafkaProducer<String, PaymentCaptured> producer = new KafkaProducer<>(props)) {
    producer.send(new ProducerRecord<>("vaultcommerce.payments.captured.v1",
        orderId, new PaymentCaptured(orderId, txnId, amount)));
    // Retry after timeout will not duplicate on the log
}