Expert · 5–6 weeks
CDC Analytics Pipeline
Capture changes from operational PostgreSQL via Debezium → Kafka → stream processing → analytics warehouse (or ClickHouse). Near-real-time dashboards without impacting OLTP.
Why this project
CDC is how modern data platforms work. Staff engineers bridge app and data teams.
Tech stack
Java 17DebeziumKafkaPostgreSQLClickHouse or BigQuerySpring Boot consumers
Skills demonstrated
- CDC
- Stream processing
- OLTP vs OLAP separation
- Schema evolution
Interview talking points
- Debezium vs polling
- Handling schema migrations
- Late-arriving data
- PII masking in pipeline
Problem statement
Capture changes from operational PostgreSQL via Debezium → Kafka → stream processing → analytics warehouse (or ClickHouse). Near-real-time dashboards without impacting OLTP.