Why this matters
- VaultCommerce fraud scoring needs sub-second reaction to click patterns, not hourly batch jobs.
- Event time vs processing time distinction prevents mis-aggregated windows during lag spikes.
- Watermarks signal 'no more late events expected' — triggers window close.
- Mental model before Kafka Streams DSL or Flink SQL.
Bounded vs unbounded
Batch: finite dataset, job completes. Stream: infinite ingress, continuous computation. VaultCommerce clickstream (vaultcommerce.analytics.clicks.v1) never 'finishes' — aggregations run forever with rolling windows.
Key points
- Unbounded stream — continuously arriving records without defined end
- Event time — timestamp when business event occurred
- Processing time — when stream processor observes the record
- Watermark — heuristic for event-time progress; advances window closure
- Late arrival — record after watermark; dropped or side-output
Time semantics
Processing time: wall clock when operator runs — simple but wrong under lag. Event time: timestamp in record — correct for business logic. VaultCommerce uses event time with 5-minute allowed lateness for order-per-minute dashboards.
VaultCommerce rollout checklist
Before promoting changes that touch vaultcommerce.analytics.clicks.v1 fraud topology, run the staging KRaft cluster (Kafka 3.7+, Schema Registry 7.x) through a 10k events/min soak test. Compare producer request latency p99 and consumer lag per group against the pre-deploy baseline. Streams are unbounded; use event time for business metrics. Document the change in the internal topic registry, attach Grafana screenshots to the change ticket, and keep an engineer on lag dashboards for 30 minutes after production rollout — roll back the service release before altering broker-level settings if lag or under-replicated partitions spike.
// VaultCommerce click event — event time in payload, not broker timestamp
public record ClickEvent(String sessionId, String productId, Instant clickedAt) {}
ProducerRecord<String, ClickEvent> record = new ProducerRecord<>(
"vaultcommerce.analytics.clicks.v1",
sessionId,
click
);
record.headers().add("event-time", click.clickedAt().toString().getBytes(UTF_8));
Quick recall
Everything you need if you only revisit this box.
- Streams are unbounded; use event time for business metrics.
- Watermarks bound lateness.
- Batch is a finite slice of a stream.
Test yourself
Answer these before moving on — recall is what makes it stick.