StreamHub uses RabbitMQ for task queues (transcode jobs with priority routing) and Kafka for event streams (viewer analytics, audit logs, and cross-service fan-out). Picking the wrong one costs operability or replay capability.
Mental model difference
| Aspect | RabbitMQ (smart broker) | Kafka (dumb broker, smart consumers) |
|---|---|---|
| Data model | Messages deleted after ack | Append-only log retained by time/size |
| Delivery | Broker pushes to consumers | Consumers pull at their offset |
| Routing | Exchanges, bindings, routing keys | Topics + partitions; no built-in routing |
| Replay | Not designed for replay | Reset offset and re-read history |
| Throughput | Thousands/sec per queue | Millions/sec per cluster |
| Best for | Task queues, RPC, priority jobs | Event streaming, analytics, audit |
Data model
RabbitMQ (smart broker)Messages deleted after ackKafka (dumb broker, smart consumers)Append-only log retained by time/sizeDelivery
RabbitMQ (smart broker)Broker pushes to consumersKafka (dumb broker, smart consumers)Consumers pull at their offsetRouting
RabbitMQ (smart broker)Exchanges, bindings, routing keysKafka (dumb broker, smart consumers)Topics + partitions; no built-in routingReplay
RabbitMQ (smart broker)Not designed for replayKafka (dumb broker, smart consumers)Reset offset and re-read historyThroughput
RabbitMQ (smart broker)Thousands/sec per queueKafka (dumb broker, smart consumers)Millions/sec per clusterBest for
RabbitMQ (smart broker)Task queues, RPC, priority jobsKafka (dumb broker, smart consumers)Event streaming, analytics, audit
Amazon MSK event pipeline
RabbitMQ patterns
Exchange types
- Direct: Route by exact routing key —
transcode.hd→ HD worker queue. - Fanout: Broadcast to all bound queues — deploy notifications to every service.
- Topic: Pattern match on routing key —
video.*.readymatchesvideo.hd.ready. - Headers: Route by message header attributes — multi-tenant routing by
tenant_id.
# RabbitMQ topology (pseudo-config)
exchanges:
- name: streamhub.tasks
type: topic
queues:
- name: transcode.hd
bindings:
- exchange: streamhub.tasks
routing_key: transcode.hd
- name: transcode.sd
bindings:
- exchange: streamhub.tasks
routing_key: transcode.sd
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 3 sections
- Kafka patterns
- When to pick which
- StreamHub hybrid architecture
Test yourself
Answer these before moving on — recall is what makes it stick.