Design StreamHub's search bar: as a user types "fort", suggest "fortnite", "fortnite ranked", "fortnite clips" ranked by popularity — under 100 ms p99 at 50K concurrent typists.
Requirements
Functional requirements
- Return top 5–10 autocomplete suggestions for a prefix (1–20 characters).
- Rank suggestions by popularity (search frequency), recency, and personalisation.
- Support fuzzy matching for typos (optional v2).
- Update suggestions as trending topics change (within minutes).
- Language-aware suggestions based on user locale.
Non-functional requirements
- Latency: p99 under 100 ms; p50 under 30 ms.
- Scale: 50K concurrent users typing; 500M autocomplete requests/day.
- Corpus size: 10M unique query strings; growing with user searches.
- Freshness: Trending queries reflected within 5 minutes.
- Availability: 99.9% — search bar is on every page.
Out of scope: Full search results page, image search, semantic/vector search.
Estimation
- 500M requests/day ÷ 86400 ≈ 5.8K QPS average, ~20K peak.
- 10M unique queries × 30 bytes avg ≈ 300 MB raw corpus — fits in memory.
- Trie with 10M strings: ~500 MB–1 GB in memory with frequency scores.
- Personalisation adds ~100 bytes × 50M users = 5 GB — store top-10 per user in Redis, not in trie.
Search & autocomplete
API design
GET /v1/search/suggest:
query:
q: "fort" # prefix
limit: 10
locale: "en-US"
response:
suggestions:
- { "text": "fortnite", "score": 98234, "type": "query" }
- { "text": "fortnite ranked", "score": 45102, "type": "query" }
- { "text": "fortnite clips", "score": 38901, "type": "query" }
-- Aggregate query log (batch job input)
CREATE TABLE search_query_log (
query_text VARCHAR(256) NOT NULL,
user_id BIGINT,
searched_at TIMESTAMPTZ NOT NULL,
locale VARCHAR(8)
);
CREATE INDEX idx_query_log_time ON search_query_log (searched_at DESC);
-- Pre-computed top queries (updated by batch job)
CREATE TABLE trending_queries (
query_text VARCHAR(256) PRIMARY KEY,
frequency BIGINT NOT NULL,
locale VARCHAR(8) NOT NULL,
updated_at TIMESTAMPTZ NOT NULL
);
High-level architecture
Data pipeline
- Query log: Every full search logged to Kafka → aggregated hourly by Spark/Flink.
- Trie builder: Batch job rebuilds in-memory trie from top 10M queries with frequency scores.
- Suggestion service: Holds trie in memory; serves prefix lookups; deployed to multiple replicas.
- Personalisation layer: Redis stores user's recent searches; merged with global trie results.
- CDN / edge cache: Cache popular prefixes ("f", "fo", "g", "ga") at CDN for global latency.
Deep dive: trie-based lookup
class TrieNode:
def __init__(self):
self.children: dict[str, TrieNode] = {}
self.top_queries: list[tuple[str, int]] = [] # pre-sorted top 10 at this prefix
def suggest(prefix: str, limit: int = 10) -> list[str]:
node = trie.root
for char in prefix.lower():
if char not in node.children:
return []
node = node.children[char]
return [q for q, _ in node.top_queries[:limit]]
Store top-K at each node during trie build — O(prefix length) lookup, no traversal of entire subtree.
Deep dive: ranking signals
| Signal | Weight | Update frequency |
|---|---|---|
| Global frequency | High | Hourly batch from query log |
| Trending boost | High | Last 1-hour frequency × 10 multiplier |
| Personal history | Medium | Real-time from Redis per user |
| Locale match | Filter | Exclude non-matching locale queries |
| Streamer name match | Medium | Index streamer display names separately |
Global frequency
WeightHighUpdate frequencyHourly batch from query logTrending boost
WeightHighUpdate frequencyLast 1-hour frequency × 10 multiplierPersonal history
WeightMediumUpdate frequencyReal-time from Redis per userLocale match
WeightFilterUpdate frequencyExclude non-matching locale queriesStreamer name match
WeightMediumUpdate frequencyIndex streamer display names separately
Deep dive: scaling and freshness
Production optimisations
- Shard trie by first character: "a-m" trie on cluster A, "n-z" on cluster B — route by prefix.
- Incremental updates: Don't rebuild full trie hourly — merge delta frequency updates into existing trie.
- CDN cache: Prefixes of length 1–2 cover 80% of traffic; TTL 5 minutes at edge.
- Fallback: If trie service is down, serve cached top-100 global queries — degrade gracefully.
Quick recall
Everything you need if you only revisit this box.
- Trie with top-K suggestions stored at each node — O(prefix length) lookup.
- Aggregate query logs hourly to compute global frequency; trending boost for recent spikes.
- Personalisation: merge user's recent searches from Redis with global trie results.
- CDN-cache popular 1–2 character prefixes; 80% of traffic hits edge.
- Shard trie by first character when corpus exceeds single-node memory.
- 500M requests/day ≈ 5.8K QPS — in-memory trie on replicated service handles this easily.
Test yourself
Answer these before moving on — recall is what makes it stick.