Distributed Cache Architecture

Client-side vs server-side caching, replication, and cache layers in production.

Advanced11 min read

Read these first

Memcached vs Redis

Why this matters

A single Redis instance caps out around 100–200K ops/sec; VaultCommerce's peak cart traffic needs a clustered topology with consistent hashing.
Client-side caching reduces network hops but introduces staleness across pods.
Production interviews ask you to diagram L1/L2 cache layers and explain what happens when a node dies.

App

RedisCache

PostgresSource of truth

Redis

App checks cache first. On miss, reads DB and populates cache.

Client-side vs server-side caching

Two distribution models

Client-side (embedded) — each app instance holds a local cache (Caffeine, Guava); zero network on hit.
Server-side (remote) — dedicated Redis/Memcached cluster; all pods share one cache pool.
Hybrid L1 + L2 — local cache in front of Redis; fastest hits, shared warmth on L2.

Java

VaultCommerce hybrid stack:

  Request → Caffeine L1 (5s TTL, per pod)
              ↓ miss
            Redis L2 (10 min TTL, cluster)
              ↓ miss
            Postgres (source of truth)

L1 catches repeated reads within a single pod during a burst. L2 shares warmth across the fleet. Invalidating L2 via pub/sub triggers L1 eviction in every pod.

Redis Cluster topology

VaultCommerce runs a 6-node Redis Cluster — 3 primaries, 3 replicas — with consistent hashing across 16,384 slots:

Java

redis_cluster:
  primaries: 3
  replicas_per_primary: 1
  hash_algorithm: crc16_slot
  client: lettuce_cluster_aware
  failover: automatic_promotion

When redis-2 fails, its replica promotes and serves slots previously owned by the failed primary — cart reads continue with sub-second failover.

Topology	Pros	Cons
Single instance	Simplest ops	SPOF, memory ceiling
Primary + Sentinel	HA failover	No horizontal scale
Redis Cluster	Scale + HA	Multi-key ops need hash tags
Memcached pool	Multi-threaded blobs	No native HA — client reroutes

Single instance
ProsSimplest ops
ConsSPOF, memory ceiling
Primary + Sentinel
ProsHA failover
ConsNo horizontal scale
Redis Cluster
ProsScale + HA
ConsMulti-key ops need hash tags
Memcached pool
ProsMulti-threaded blobs
ConsNo native HA — client reroutes

Cache layers in production

Aspect	L1 (in-process)	L2 (Redis cluster)
Latency	~microseconds	~1 ms LAN
Consistency	Stale across pods	Shared, near-uniform
Invalidation	Pub/sub or short TTL	DEL key or CDC event
VaultCommerce data	Category tree, feature flags	Products, sessions, carts

Latency
L1 (in-process)~microseconds
L2 (Redis cluster)~1 ms LAN
Consistency
L1 (in-process)Stale across pods
L2 (Redis cluster)Shared, near-uniform
Invalidation
L1 (in-process)Pub/sub or short TTL
L2 (Redis cluster)DEL key or CDC event
VaultCommerce data
L1 (in-process)Category tree, feature flags
L2 (Redis cluster)Products, sessions, carts

Java

@Cacheable(value = "categories", cacheManager = "caffeineManager")
public CategoryTree getCategories() { ... }

// L2 populated separately on product reads via cache-aside

Hot key mitigation

A viral product can overload one Redis slot:

Hot key strategies

Local replication — app caches hot key in L1 with 2-second TTL.
Read replicas — Redis 7+ supports read replicas per shard for read-heavy keys.
Key splitting — product:8842:copy-{0..3} with random read selection.
Pre-warming — load hot keys before marketing campaigns go live.

VaultCommerce detected a hot key when a celebrity endorsement spiked product:VC-SNEAKER-01 to 400K reads/sec on one slot. L1 replication plus four key copies spread load to 100K per copy.

Observability and failure modes

Monitor per-node memory, connected clients, instantaneous_ops_per_sec, and cluster cluster_state. Run chaos tests: kill a primary during load tests and verify client libraries refresh slot maps without manual intervention.

Quick recall

Everything you need if you only revisit this box.

Client-side (L1) is fastest but per-pod; server-side (L2) shares warmth across the fleet.
VaultCommerce uses Caffeine L1 + Redis Cluster L2 + Postgres.
Redis Cluster combines consistent hashing, replication, and automatic failover.
Hot keys overload single slots — replicate locally, split keys, or pre-warm.
Invalidate L1 when L2 changes; monitor cluster health and test failover regularly.

Test yourself

Answer these before moving on — recall is what makes it stick.