Skip to main content
PreparationZone Logo
PrepZone

Rate Limiter Design

Token bucket, sliding window and distributed counters that protect APIs from abuse.

Pass checkpoint below

StreamHub's public API allows 100 requests per minute on the free tier and 1,000 on Pro. Without a distributed limiter, one abusive scraper could starve legitimate viewers — and without atomic counters, two gateway pods could each allow 100, letting 200 through.

Requirements

Functional requirements

  • Enforce quotas keyed by user ID, API key, IP, or tenant.
  • Support multiple tiers (Free 100/min, Pro 1K/min, Enterprise custom).
  • Return HTTP 429 with Retry-After and rate-limit headers.
  • Allow hot-reload of rules without gateway restart.

Non-functional requirements

  • Latency: add less than 10 ms per request.
  • Accuracy: ~99% in distributed mode; slight over-count is acceptable.
  • Scale: 15K+ peak QPS across many gateway pods globally.
  • Failure mode: fail-open for public reads; fail-closed for payment endpoints.

Back-of-the-envelope estimation

  • 10M DAU × 50 requests/day = 500M requests/day → ~5.8K avg QPS, ~15K peak.
  • Counter state: (user_id, minute_bucket) → integer. ~10M users × 60 buckets ≈ 600M keys/hour.
  • Redis INCR is O(1); one shard handles 100K+ ops/sec; cluster scales linearly.
Request
Token bucket100 req/min refill
API

No tokens → 429 Too Many Requests.

Tokens refill at a fixed rate; each request consumes one token.

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 3 sections

  • Algorithm comparison
  • Distributed implementation with Redis
  • High-level architecture

Test yourself

Answer these before moving on — recall is what makes it stick.