Why this matters
- Cross-shard transfers cannot use 2PC without sacrificing availability — orchestrated sagas are ShardPay's production path.
- A durable saga log gives on-call a visible state machine instead of inferring progress from scattered logs.
- Every forward step must have a defined compensating action or be safely retryable — interviewers probe this relentlessly.
- Orchestration trades a coordinator dependency for debuggability; ShardPay accepts that trade for its highest-value payment flows.
Orchestrated cross-shard transfer
ShardPay's transfer saga has four steps: validate fraud score, debit source shard, credit destination shard, emit settlement event. The orchestrator persists state after each step so a crash mid-saga resumes from the last completed step, not from scratch.
When merchant M-8812 sends $1,200 from account A-US-7 (shard 3) to A-EU-2 (shard 9), the orchestrator creates saga instance SAGA-9f3a in state STARTED, calls fraud service, then transitions to DEBIT_PENDING. Only after a successful debit does it attempt credit — never both in one atomic operation.
Key points
- Orchestrator — a dedicated service (or module) that owns saga state and invokes participants in order. ShardPay's
TransferOrchestratorruns on Java 17 with a Postgres-backed state table and leader election via etcd. - Saga instance — one row per in-flight business transaction with current state, step outputs, and timestamps. ShardPay indexes sagas by
transferIdso support can look upSAGA-9f3afrom a merchant ticket. - Compensation — a semantic undo of a completed forward step, not a database rollback. If credit fails after debit succeeds, ShardPay runs
CompensateDebit— a new transaction that credits the source account back with reasonSAGA_ROLLBACK. - Timeout per step — if credit does not complete within 30 seconds, the orchestrator marks the step failed and triggers compensation. ShardPay alerts when >50 sagas exceed step timeout in 5 minutes.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 2 sections
- Saga state machine
- Recovery and idempotency
Test yourself
Answer these before moving on — recall is what makes it stick.