Why this matters
- Missing timeouts caused thread pool exhaustion at ShardPay when the fraud vendor hung for 90 seconds during a regional network glitch.
- End-to-end latency budgets force explicit trade-offs instead of every dependency getting "30 seconds just in case."
- Interviewers hand you a 200ms SLA and ask you to slice budget across five microservices.
- Propagating deadlines prevents downstream work from running after the client already disconnected.
End-to-end budget anatomy
ShardPay's synchronous transfer path targets p99 ≤ 200ms from API gateway to response. That 200ms is not per service — it is the entire user-visible window. The gateway reserves 20ms for TLS and routing, 30ms for auth token validation, 100ms for ledger debit/credit on the shard, 50ms for fraud scoring, and 20ms slack for serialization and network jitter.
If fraud consistently consumes 50ms, ledger cannot borrow its unused slack without violating p99. Budgets are negotiated at design time and enforced in code, not renegotiated per request.
Walkthrough: budget exhausted mid-chain
A transfer hits the gateway at T+0ms. Auth completes at T+25ms (5ms over, but within hard cap). Ledger starts with 175ms remaining on the propagated deadline. Shard 7 is slow; ledger returns at T+120ms with 80ms left. Fraud starts with min(local_cap=50ms, remaining=80ms) = 50ms. Fraud's vendor times out at T+170ms. ShardPay returns 202 Accepted with status=pending_fraud instead of waiting another 30ms and breaching the 200ms SLA.
Key points
- End-to-end budget — the total SLA clock started when the client sent the request. ShardPay's gateway starts a 200ms
Deadlineon every POST /transfers and refuses to enqueue work when fewer than 10ms remain — fail fast rather than guaranteed timeout. - Per-hop timeout — each dependency gets a hard cap smaller than the remaining budget. Ledger's 100ms cap prevents a slow shard from consuming fraud's allocation; ops tune caps from distributed trace percentiles quarterly.
- Context deadline propagation — gRPC and W3C
traceparentbaggage carrydeadline_msso each hop knows remaining time. ShardPay's Java services readDeadline.afterNow()and pass shortened stubs to downstream clients automatically. - Graceful partial response — when budget is exhausted before the full pipeline completes, return a definitive async state (
202,transferId, poll URL) instead of504 Gateway Timeout. ShardPay reduced duplicate client retries by 40% after replacing bare 504s with structured pending responses. - Budget vs timeout — budget is the product SLA; timeout is the enforcement mechanism per hop. A hop may timeout at 50ms even if 80ms remains globally, to preserve slack for mandatory downstream steps.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 2 sections
- Propagating deadlines in Java
- Measuring and tuning budgets
Test yourself
Answer these before moving on — recall is what makes it stick.