PrepZone Logo
PrepZone

Topic Design, Naming, and Partition Count

vaultcommerce.orders.placed.v1 — naming conventions, partition count, and replication factor for production topics.

Why this matters

  • VaultCommerce enforces vaultcommerce.{domain}.{event}.v{N} — grep-friendly, ACL-scoped, versioned.
  • Partition count cannot shrink; increasing partitions does not rebalance existing keys.
  • Separate topics per event type enable independent retention and consumer scaling.
  • Bad naming causes ACL sprawl and on-call confusion during incidents.
ProducersOrder, payment services
Broker cluster3+ brokers, KRaft
Consumer groupsInventory, analytics
Topic: vaultcommerce.orders.placed.v1
Partition 0Leader on broker-1
Partition 1Leader on broker-2
Partition 2Leader on broker-3
Producers write to topic partitions on brokers. Consumers read via consumer groups.

Naming conventions

Domain boundaries map to ACL prefixes: vaultcommerce.orders.*, vaultcommerce.payments.*. Version suffix (v1) allows parallel old/new consumers during schema migration. Internal CDC topics use vaultcommerce.cdc.postgres.{table}.

Key points

  • Topic registry — catalog of owners, retention, partition count, schemas
  • Version suffix — v1/v2 parallel run during consumer upgrades
  • Partition count — upper bound on consumer parallelism; plan for peak × 1.5
  • Replication factor — 3 for production; 1 only in local compose
  • Topic cleanup.policy — delete vs compact per use case

Partition and RF planning

Orders: 12 partitions, RF=3. Customer profile changelog: 6 partitions, compacted. Dead-letter: 3 partitions, RF=3, 30-day retention. Document decisions in the topic registry — VaultCommerce's Terraform module enforces naming regex.

VaultCommerce rollout checklist

Before promoting changes that touch the VaultCommerce order and payment event backbone, run the staging KRaft cluster (Kafka 3.7+, Schema Registry 7.x) through a 10k events/min soak test. Compare producer request latency p99 and consumer lag per group against the pre-deploy baseline. Name with domain.event.version pattern. Document the change in the internal topic registry, attach Grafana screenshots to the change ticket, and keep an engineer on lag dashboards for 30 minutes after production rollout — roll back the service release before altering broker-level settings if lag or under-replicated partitions spike.

Java
# VaultCommerce topic registry excerpt (Terraform / GitOps)
topics:
  - name: vaultcommerce.orders.placed.v1
    partitions: 12
    replication_factor: 3
    config:
      retention.ms: 604800000        # 7 days
      min.insync.replicas: 2
  - name: vaultcommerce.customers.profile.v1
    partitions: 6
    replication_factor: 3
    config:
      cleanup.policy: compact
      min.compaction.lag.ms: 3600000

Quick recall

Everything you need if you only revisit this box.

  1. Name with domain.event.version pattern.
  2. Partition count is hard to reduce later.
  3. Match RF and min ISR to durability requirements.

Test yourself

Answer these before moving on — recall is what makes it stick.