Skip to main content
PreparationZone Logo
PrepZone

Kafka Streams: Topology and DSL

Build a VaultCommerce fraud-detection topology with map, filter, groupBy, and aggregate operators.

Pass checkpoint below

Why this matters

  • VaultCommerce fraud-detection runs as an embedded Kafka Streams app in the payment service JVM.
  • No separate cluster — scales with application pods; state in local RocksDB.
  • DSL (stream.groupByKey().aggregate()) covers 80% of VaultCommerce use cases.
  • Exactly-once v2 processing guarantee with processing.guarantee=exactly_once_v2.
Source topic
groupBy key
Sink topic
Source → map/filter → groupBy → aggregate → sink. State stored in RocksDB with changelog backup.

Topology building blocks

KStream for record-per-event; KTable for changelog (latest per key). vaultcommerce.analytics.clicks.v1 → filter bot traffic → groupBy sessionId → count → vaultcommerce.fraud.signals.v1. Application ID vaultcommerce-fraud-detector defines consumer group and state store namespace.

Key points

  • application.id — consumer group + state store prefix; unique per topology
  • KStream — unbounded sequence of records
  • KTable — changelog stream materialized as key-value table
  • Topology — DAG of processors wired to Kafka topics
  • exactly_once_v2 — transactional read-process-write guarantee

The rest of this guide is part of Pro

Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.

Still to come · 2 sections

  • Deployment model
  • VaultCommerce rollout checklist

Test yourself

Answer these before moving on — recall is what makes it stick.