Eviction, TTL, and Cache Stampede

LRU policies, time-to-live, and preventing thundering herd on hot keys.

Intermediate10 min read

Read these first

Caching Patterns

Why this matters

VaultCommerce's homepage hero product key expiring at noon once took Postgres from 40% to 98% CPU in 30 seconds — a textbook stampede during a flash sale.
LRU sounds simple until you realise one large key evicts thousands of small session tokens.
Interviews ask how you'd protect a hot key beyond "add a TTL."

App

RedisCache

PostgresSource of truth

Redis

App checks cache first. On miss, reads DB and populates cache.

Eviction policies

Common eviction algorithms

LRU (Least Recently Used) — evict the key untouched longest; Redis default for maxmemory-policy allkeys-lru.
LFU (Least Frequently Used) — evict rarely accessed keys; better when access is bursty.
TTL-based — expire keys on schedule regardless of access frequency.
Random — simple but unpredictable; rare in production caches.

Java

CONFIG SET maxmemory 4gb
CONFIG SET maxmemory-policy allkeys-lru

VaultCommerce runs Redis with allkeys-lru and monitors evicted_keys — a sustained spike means memory is undersized or keys are too large.

TTL design

Time-to-live balances freshness against database load:

Java

// Jittered TTL — prevent synchronized expiry
int baseTtl = 600; // 10 minutes
int jitter = ThreadLocalRandom.current().nextInt(60);
redis.setex(key, baseTtl + jitter, value);

Fixed TTL on every product:flash-sale-hero key expiring at the same second causes coordinated misses. Jitter spreads expirations across a 60-second window — VaultCommerce applies jitter to all catalog keys above 1M daily hits.

Data type	TTL	Rationale
Product detail	10 min + jitter	CDC invalidation handles urgent changes
Session	30 min sliding	Refresh on each request
Rate limit counter	60 sec fixed	Window must align to minute boundary
Flash sale banner	30 sec + lock	Short TTL, stampede-protected

Product detail
TTL10 min + jitter
RationaleCDC invalidation handles urgent changes
Session
TTL30 min sliding
RationaleRefresh on each request
Rate limit counter
TTL60 sec fixed
RationaleWindow must align to minute boundary
Flash sale banner
TTL30 sec + lock
RationaleShort TTL, stampede-protected

Cache stampede mechanics

Java

T=0:  key expires
T=1:  10,000 requests → cache MISS
T=1:  10,000 identical Postgres queries fire
T=3:  database saturated, timeouts cascade

The fix is ensuring only one request rebuilds the value while others wait or serve stale data.

Stampede prevention techniques

VaultCommerce defences

Mutex / lock — first miss acquires SETNX rebuild:product:SKU; others spin or return stale.
Stale-while-revalidate — serve expired value while async refresh runs in background.
Probabilistic early refresh — recompute before TTL if key is within random early window.
Request coalescing — single-flight pattern deduplicates in-flight loads per key.

Java

public Product getWithLock(String sku) {
    Product p = redis.get("product:" + sku);
    if (p != null) return p;

    if (redis.setnx("lock:product:" + sku, "1", Duration.ofSeconds(5))) {
        try {
            p = productRepo.findBySku(sku);
            redis.setex("product:" + sku, 600, p);
            return p;
        } finally {
            redis.del("lock:product:" + sku);
        }
    }
    Thread.sleep(50);
    return getWithLock(sku); // retry — another thread populated cache
}

For the flash-sale hero SKU, VaultCommerce pre-warms the cache five minutes before expiry and uses stale-while-revalidate so shoppers never see a blank page.

Quick recall

Everything you need if you only revisit this box.

Eviction policies (LRU, LFU) decide what leaves when memory is full.
Jittered TTLs prevent thousands of keys expiring in the same second.
Cache stampede: mass simultaneous misses overload the database.
Prevent with mutex locks, stale-while-revalidate, or probabilistic early refresh.
VaultCommerce pre-warms hot keys before flash sales and monitors evicted_keys.

Test yourself

Answer these before moving on — recall is what makes it stick.