Skip to main content
PreparationZone Logo
PrepZone

RabbitMQ and Kafka Patterns

Point-to-point versus log-based messaging — when to pick each and how to configure them.

Pass checkpoint below

StreamHub uses RabbitMQ for task queues (transcode jobs with priority routing) and Kafka for event streams (viewer analytics, audit logs, and cross-service fan-out). Picking the wrong one costs operability or replay capability.

Mental model difference

AspectRabbitMQ (smart broker)Kafka (dumb broker, smart consumers)
Data modelMessages deleted after ackAppend-only log retained by time/size
DeliveryBroker pushes to consumersConsumers pull at their offset
RoutingExchanges, bindings, routing keysTopics + partitions; no built-in routing
ReplayNot designed for replayReset offset and re-read history
ThroughputThousands/sec per queueMillions/sec per cluster
Best forTask queues, RPC, priority jobsEvent streaming, analytics, audit
  • Data model

    RabbitMQ (smart broker)Messages deleted after ack
    Kafka (dumb broker, smart consumers)Append-only log retained by time/size
  • Delivery

    RabbitMQ (smart broker)Broker pushes to consumers
    Kafka (dumb broker, smart consumers)Consumers pull at their offset
  • Routing

    RabbitMQ (smart broker)Exchanges, bindings, routing keys
    Kafka (dumb broker, smart consumers)Topics + partitions; no built-in routing
  • Replay

    RabbitMQ (smart broker)Not designed for replay
    Kafka (dumb broker, smart consumers)Reset offset and re-read history
  • Throughput

    RabbitMQ (smart broker)Thousands/sec per queue
    Kafka (dumb broker, smart consumers)Millions/sec per cluster
  • Best for

    RabbitMQ (smart broker)Task queues, RPC, priority jobs
    Kafka (dumb broker, smart consumers)Event streaming, analytics, audit

Amazon MSK event pipeline

produceCOMPUTE
EKS APIorder placed
INTEGRATION
Amazon MSKorders.placed.v1
COMPUTE
InventoryEKS worker
COMPUTE
Email svcEKS worker
ANALYTICS
AnalyticsFlink / EMR
API publishes events; worker fleets scale independently on consumer lag.

RabbitMQ patterns

Exchange types

  • Direct: Route by exact routing key — transcode.hd → HD worker queue.
  • Fanout: Broadcast to all bound queues — deploy notifications to every service.
  • Topic: Pattern match on routing key — video.*.ready matches video.hd.ready.
  • Headers: Route by message header attributes — multi-tenant routing by tenant_id.
Java
# RabbitMQ topology (pseudo-config)
exchanges:
  - name: streamhub.tasks
    type: topic
queues:
  - name: transcode.hd
    bindings:
      - exchange: streamhub.tasks
        routing_key: transcode.hd
  - name: transcode.sd
    bindings:
      - exchange: streamhub.tasks
        routing_key: transcode.sd

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 3 sections

  • Kafka patterns
  • When to pick which
  • StreamHub hybrid architecture

Test yourself

Answer these before moving on — recall is what makes it stick.