When to Distribute (and When Not)

The distribution tax

Every network hop adds latency, failure modes, and versioning problems. ShardPay's monolith handled 2,000 transfers/sec on one beefy machine with a single Postgres instance and one deploy pipeline. After splitting into three services, the team needed Kubernetes, distributed tracing, contract tests, and a three-team on-call rotation before matching that throughput — and p99 latency was higher for six months.

The distribution tax is real and recurring. It is not paid once at the split; it is paid on every feature that crosses a service boundary, every incident that spans three log systems, and every deploy that must coordinate schema versions across teams.

When ShardPay stayed monolithic

For its first 18 months, ShardPay was a modular monolith: separate Java packages for ledger, API, and admin, one WAR file, one database. The team was eight engineers shipping weekly. Cross-cutting changes (add a field to transfers) touched one repo, one migration, one deploy. Latency was predictable because there were no network hops between ledger logic and fraud rules — just in-process calls.

The monolith was not "legacy." It was the correct architecture until proven constraints appeared. Premature microservices would have slowed feature delivery without solving a real bottleneck.

When ShardPay split

Three independent scaling profiles justified three services. Fraud scoring needed GPU nodes and a Python runtime with a different release cadence than the Java ledger. The ledger needed CP consistency and dedicated I/O tuning. Notifications were AP, bursty, and tolerable at seconds of delay. None of these profiles could share one machine sizing or one deploy schedule without waste or risk.

Walkthrough: the ledger split decision

ShardPay's ledger hit 32 cores and disk I/O saturation at 4,500 transfers/sec. Vertical scaling to a larger instance bought three months. Horizontal sharding required extracting the ledger into its own service with a stable RPC contract, so API pods could scale independently. The split took six weeks; the decision took six months of metrics proving the monolith could not shard in place.

// Before split — in-process call inside the monolith
@Service
public class TransferService {
    private final LedgerRepository ledger;
    private final FraudRulesEngine fraud;  // same JVM, same transaction boundary

    public TransferResult transfer(TransferRequest req) {
        fraud.check(req);                          // ~5ms, no network
        return ledger.debitAndCredit(req);         // single DB transaction
    }
}

// After split — network hop, separate failure domain, saga for cross-shard
@Service
public class TransferOrchestrator {
    private final LedgerClient ledger;           // gRPC, 80ms budget
    private final FraudClient fraud;             // separate deploy, GPU pool

    public TransferResult transfer(TransferRequest req) {
        fraud.check(req);                          // RPC, may timeout independently
        return ledger.transfer(req);               // saga if accounts cross shards
    }
}

Signs you are ready to distribute

ShardPay used a checklist before each split: (1) a measured bottleneck that cannot be fixed in the monolith, (2) a team boundary that matches the service boundary, (3) a stable domain API that will not change weekly, and (4) operational readiness for independent deploys and monitoring. Missing any item meant staying in the modulith longer.

Conversely, signs you are not ready: "we want microservices," "other companies do it," or "the monolith feels messy." Messy code splits into messy services. Clean module boundaries in a modulith are cheaper to maintain than distributed spaghetti.