Kafka Streams: Topology and DSL

Topology building blocks

KStream for record-per-event; KTable for changelog (latest per key). vaultcommerce.analytics.clicks.v1 → filter bot traffic → groupBy sessionId → count → vaultcommerce.fraud.signals.v1. Application ID vaultcommerce-fraud-detector defines consumer group and state store namespace.

Deployment model

One stream thread per pod typically; num.stream.threads for parallelism within JVM. VaultCommerce runs 4 pods × 1 thread = 4 tasks (must match partition count). Rebalance on pod crash restores state from changelog.

VaultCommerce rollout checklist

Before promoting changes that touch vaultcommerce.analytics.clicks.v1 fraud topology, run the staging KRaft cluster (Kafka 3.7+, Schema Registry 7.x) through a 10k events/min soak test. Compare producer request latency p99 and consumer lag per group against the pre-deploy baseline. Kafka Streams embeds in your JVM. Document the change in the internal topic registry, attach Grafana screenshots to the change ticket, and keep an engineer on lag dashboards for 30 minutes after production rollout — roll back the service release before altering broker-level settings if lag or under-replicated partitions spike.

// VaultCommerce fraud topology — Kafka Streams 3.7+
StreamsBuilder builder = new StreamsBuilder();
KStream<String, ClickEvent> clicks = builder.stream("vaultcommerce.analytics.clicks.v1");

clicks.filter((k, v) -> v.productId() != null)
    .groupByKey()
    .windowedBy(TimeWindows.ofSizeWithNoGrace(Duration.ofMinutes(5)))
    .count()
    .toStream()
    .mapValues((windowedKey, count) -> new FraudSignal(windowedKey.key(), count))
    .to("vaultcommerce.fraud.signals.v1");

KafkaStreams streams = new KafkaStreams(builder.build(), streamsConfig);
streams.start();