Idempotency Patterns

Safe retries with idempotency keys so duplicate requests never double-charge or double-send.

IntermediateTheory10 min read

Read these first

Rate Limiter Design

When StreamHub's subscription checkout times out, the client retries the same POST. Without an idempotency key, the user gets charged twice. With one, the second request returns the original receipt — safe and expected.

Why retries happen

Sources of duplicate requests

Client retries: Mobile apps retry on timeout; users double-tap submit buttons.
Gateway retries: Load balancers replay requests when upstream returns 502/503.
Message queue redelivery: At-least-once brokers redeliver unacknowledged messages.
Webhook retries: Partners retry failed callback deliveries with exponential backoff.

HTTP GET, PUT, and DELETE are naturally idempotent. POST is not — you must design idempotency explicitly.

Idempotency key flow (checkout)

Client retry after timeout hits dedup store — same key returns original receipt.

Idempotency key pattern

How the pattern works

Client generates a unique key (UUID) per logical operation and sends it in a header.
Server checks a store: if key exists, return the stored response; if not, process and store.
Keys expire after 24–72 hours — long enough for retry windows, short enough to limit storage.

Java

POST /v1/subscriptions HTTP/1.1
Authorization: Bearer eyJ...
Idempotency-Key: 7c9e6679-7425-40de-944b-e07fc1f90ae7
Content-Type: application/json

{"plan": "pro", "streamer_id": "sh_8821"}

Java

CREATE TABLE idempotency_keys (
    key           VARCHAR(64) PRIMARY KEY,
    user_id       BIGINT NOT NULL,
    request_hash  VARCHAR(64) NOT NULL,   -- detect key reuse with different body
    response_code INT NOT NULL,
    response_body JSONB NOT NULL,
    created_at    TIMESTAMPTZ DEFAULT now(),
    expires_at    TIMESTAMPTZ NOT NULL
);

CREATE INDEX idx_idempotency_expires ON idempotency_keys (expires_at);

Server-side flow

Java

def create_subscription(req: Request) -> Response:
    key = req.headers.get("Idempotency-Key")
    if not key:
        raise BadRequest("Idempotency-Key required")

    existing = db.get_idempotency(key, req.user_id)
    if existing:
        if existing.request_hash != hash_body(req.body):
            raise Conflict("Key reused with different payload")
        return Response(existing.response_code, existing.response_body)

    # Acquire lock to prevent concurrent duplicate processing
    with redis.lock(f"idempotency:{key}", timeout=30):
        result = process_subscription(req.body)
        db.save_idempotency(key, req.user_id, hash_body(req.body), result)
        return result

Idempotency without a key store

Aspect	Natural idempotency	Idempotency key store
Mechanism	Upsert by unique constraint (order_id)	Client-supplied UUID in header + response cache
Best for	Operations with natural unique IDs	POST creates without client-known ID
Storage	Business table enforces uniqueness	Dedicated idempotency table or Redis
StreamHub example	Follow relationship (user_id, streamer_id) unique	Subscription checkout POST

Mechanism
Natural idempotencyUpsert by unique constraint (order_id)
Idempotency key storeClient-supplied UUID in header + response cache
Best for
Natural idempotencyOperations with natural unique IDs
Idempotency key storePOST creates without client-known ID
Storage
Natural idempotencyBusiness table enforces uniqueness
Idempotency key storeDedicated idempotency table or Redis
StreamHub example
Natural idempotencyFollow relationship (user_id, streamer_id) unique
Idempotency key storeSubscription checkout POST

Java

-- Natural idempotency via unique constraint
INSERT INTO follows (user_id, streamer_id)
VALUES (42, 8821)
ON CONFLICT (user_id, streamer_id) DO NOTHING;

Distributed considerations

Production checklist

Lock before process: Two concurrent retries with the same key must not both execute — use Redis lock or DB SELECT FOR UPDATE.
Store response, not just flag: Return the exact same response body and status code on replay.
Hash the request body: Reject key reuse with a different payload (HTTP 409 Conflict).
TTL cleanup: Cron job or Redis expiry deletes keys after the retry window closes.
Scope keys per user: user_id + key prevents cross-user key collision.

Quick recall

Everything you need if you only revisit this box.

Networks retry — design every POST that mutates state to be safely repeatable.
Idempotency-Key header: client UUID, server stores response, replays on duplicate.
Lock during processing so concurrent retries don't double-execute.
Natural idempotency via unique DB constraints works when the entity ID is known upfront.
Always hash the request body and reject key reuse with different payloads.
Expire keys after 24–72 hours; scope keys per user or tenant.

Test yourself

Answer these before moving on — recall is what makes it stick.