Why this matters
- Causal consistency requires tracking dependencies between events, not just timestamps.
- Vector clocks grow with node count — use them in bounded domains like a single shard's in-memory buffer, not cluster-wide.
- Interviewers distinguish happened-before from concurrent and ask how you resolve ties.
- Misordering audit events triggers false compliance alerts and expensive manual reconciliation.
Lamport logical clocks
Each ShardPay ledger node maintains a Lamport counter. On a local event the counter increments. Before sending a message the node increments again and attaches the timestamp. On receive the node sets its counter to max(local, received) + 1.
If event A's Lamport timestamp is less than event B's, then A might have happened before B — but equality does not prove concurrency; it may mean a tie that requires a secondary key. ShardPay breaks ties with node_id lexicographic order so every pair has a total sort for audit export.
Walkthrough: cross-shard transfer chain
Account A on shard 1 sends funds to account B on shard 2, which immediately forwards to account C on shard 3. Three nodes emit audit events. Lamport timestamps propagate through the RPC chain: debit on shard 1 gets (5, node-1), credit on shard 2 receives (5) and emits (6, node-2), debit on shard 3 emits (7, node-3). Compliance export sorts 5 < 6 < 7 and sees the correct causal chain even though wall clocks on the three nodes differ by 400ms.
Key points
- Lamport timestamp — a single integer per node that respects happened-before: if A → B causally, then L(A) < L(B). ShardPay stamps every audit row with Lamport time so cross-shard exports sort causally without a central clock; ties use
node_id. - Happened-before (→) — a partial order: A → B if A caused B directly or transitively. ShardPay's dispute tooling filters "all events that happened-before this chargeback" using Lamport order plus transfer-id linkage.
- Concurrent events — neither A → B nor B → A; Lamport timestamps may be equal or incomparable without extra structure. Two unrelated merchant payouts on different shards are concurrent; ShardPay does not force a global order between them.
- Vector clock — an array of counters, one per node, that detects true concurrency: if neither vector dominates the other, events are concurrent. ShardPay uses 8-element vectors inside a single shard's conflict buffer for optimistic balance updates before commit.
- Causal delivery — a messaging guarantee that if send(A) happens-before send(B), consumers observe A before B. ShardPay's audit relay tags Kafka headers with Lamport time so downstream SIEM consumers respect causal order per partition key.
Vector clocks for concurrency detection
Lamport clocks cannot prove concurrency — equal timestamps might mean concurrent events or a tie. Vector clocks fix this by tracking each node's latest counter. Event A's vector [2,1,0] dominates [1,1,0], so A happened-after the other. Vectors [2,1,0] and [1,2,0] are incomparable — truly concurrent.
ShardPay limits vector clocks to per-shard replication buffers (three replicas, three slots). Cluster-wide vectors with 256 shards would be impractical. For global ordering ShardPay falls back to Snowflake IDs or the Raft sequencer described in the sequencers article.
Walkthrough: detecting a write conflict
Two API requests hit different replicas of the same balance row within the same millisecond. Replica R1 applies a $100 debit; R2 applies a $50 debit based on stale state. Vector clocks on the row versions show neither dominates — a true conflict. ShardPay's merge policy rejects the stale write and returns 409 Conflict so the client refreshes balance and retries with a new idempotency key.
// Lamport clock on a ShardPay ledger node (Java 17)
public final class LamportClock {
private long counter;
public synchronized long tick() {
return ++counter;
}
public synchronized long onReceive(long incoming) {
counter = Math.max(counter, incoming) + 1;
return counter;
}
public synchronized long tickAndSend() {
return ++counter; // attach to outbound RPC or audit event
}
}
Practical limits and merge rules
Concurrent events need application-level merge rules. Lamport and vector clocks tell you that two writes raced; they do not tell you which balance wins. ShardPay uses last-writer-wins only for non-financial metadata (merchant display names). Financial balances use compare-and-set on version vectors — stale versions fail fast.
For audit logs where concurrent events are acceptable, ShardPay sorts by Lamport then node_id and exports without merge. For ledger state where concurrency is a bug, vector detection triggers explicit conflict handling.
Quick recall
Everything you need if you only revisit this box.
- Lamport timestamps respect happened-before but do not detect true concurrency — add tie-breakers or vector clocks.
- Vector clocks detect concurrency in bounded domains; ShardPay uses them per-shard, not cluster-wide.
- Clocks reveal ordering and conflicts; merge policy is always application-specific for financial data.
Test yourself
Answer these before moving on — recall is what makes it stick.