Messaging & In-Memory Data interview questions

What is Apache Kafka and what problems does it solve in distributed systems?

Easy · Messaging & In-Memory Data, Kafka, Fresher

Explain Kafka’s core architecture: brokers, topics, partitions, leaders and followers.

Medium · Messaging & In-Memory Data, Kafka, Senior

How does Kafka achieve durability and high availability?

Medium · Messaging & In-Memory Data, Kafka, Senior

What is a partition key and how does partitioning affect ordering and scalability?

Medium · Messaging & In-Memory Data, Kafka, Senior

Explain the role of the Kafka controller.

Medium · Messaging & In-Memory Data, Kafka, Senior

How do producers and consumers work in Kafka? Describe producer acks and consumer groups.

Medium · Messaging & In-Memory Data, Kafka, Senior

What is the difference between consumer offset auto-commit and manual commit? When to use each?

Easy · Messaging & In-Memory Data, Kafka, Fresher

Explain at-least-once, at-most-once, and exactly-once delivery semantics—how to implement them in Kafka?

Hard · Messaging & In-Memory Data, Kafka, Staff

What is idempotence for Kafka producers and how does it help with exactly-once semantics?

Hard · Messaging & In-Memory Data, Kafka, Staff

How does Kafka handle replication and leader election for partitions?

Medium · Messaging & In-Memory Data, Kafka, Senior

What is ISR (in-sync replica) and why is it important?

Easy · Messaging & In-Memory Data, Kafka, Fresher

How do you tune Kafka producer and consumer performance (batch.size, linger.ms, fetch.min.bytes, max.poll.records)?

Medium · Messaging & In-Memory Data, Kafka, Senior

Explain log compaction vs log retention. When would you use each?

Medium · Messaging & In-Memory Data, Kafka, Senior

What is Kafka Streams and how is it different from Kafka Consumer API?

Easy · Messaging & In-Memory Data, Kafka, Fresher

Explain the core concepts of Kafka Streams: KStream, KTable, state stores, and joins.

Medium · Messaging & In-Memory Data, Kafka, Senior

How do you implement windowed aggregations in Kafka Streams?

Medium · Messaging & In-Memory Data, Kafka, Senior

What is Kafka Connect? How does it simplify data integration?

Medium · Messaging & In-Memory Data, Kafka, Senior

Explain source vs sink connectors. Give examples for AWS integrations (S3, DynamoDB).

Medium · Messaging & In-Memory Data, Kafka, Senior

How do you ensure schema compatibility in Kafka? Discuss Schema Registry and Avro/JSON Schema/Protobuf.

Medium · Messaging & In-Memory Data, Kafka, Staff

How does Kafka handle backpressure and slow consumers? Strategies to mitigate.

Medium · Messaging & In-Memory Data, Kafka, Staff

How do you design topics and partitioning strategies for multi-tenant systems?

Hard · Messaging & In-Memory Data, Kafka, Architect

What are common Kafka monitoring metrics to track cluster health and performance?

Easy · Messaging & In-Memory Data, Kafka, Fresher

How do you perform rebalancing and what are the pitfalls (stability, consumer downtime)?

Hard · Messaging & In-Memory Data, Kafka, Staff

How do you secure Kafka (TLS, SASL, ACLs) in production?

Hard · Messaging & In-Memory Data, Kafka, Architect

Describe a real-world example: designing an event-driven order processing pipeline with Kafka—what are the key design decisions?

Hard · Messaging & In-Memory Data, Kafka, Architect

What is Redis and where is it typically used in application architectures?

Easy · Messaging & In-Memory Data, Redis, Fresher

Explain the differences between in-memory cache (Redis) and distributed caches/databases.

Medium · Messaging & In-Memory Data, Redis, Senior

What are common Redis data structures and real-world use cases for each (strings, hashes, lists, sets, sorted sets)?

Hard · Messaging & In-Memory Data, Redis, Staff

What caching strategies exist (cache-aside, read-through, write-through, write-back)? Which to choose when?

Easy · Messaging & In-Memory Data, Redis, Fresher

How do you design a cache-aside pattern with Redis in a Spring Boot application?

Hard · Messaging & In-Memory Data, Redis, Staff

How do you implement cache invalidation strategies? (time-based TTL, explicit invalidation, versioning)

Medium · Messaging & In-Memory Data, Redis, Senior

How do you handle cache stampede, cache avalanche, and cache penetration?

Medium · Messaging & In-Memory Data, Redis, Staff

Explain Redis persistence options: RDB snapshots and AOF. Trade-offs between them.

Hard · Messaging & In-Memory Data, Redis, Staff

What are Redis eviction policies (LRU, LFU, volatile-*) and when to use each?

Easy · Messaging & In-Memory Data, Redis, Fresher

How do you configure and use Redis clustering for high availability and horizontal scaling?

Medium · Messaging & In-Memory Data, Redis, Senior

Explain Redis replication (master-replica) and failover behavior.

Medium · Messaging & In-Memory Data, Redis, Senior

How does Redis Sentinel work and when should you use it?

Medium · Messaging & In-Memory Data, Redis, Senior

What is Redis Cluster hash slot allocation and how does it influence resharding?

Medium · Messaging & In-Memory Data, Redis, Senior

How do you implement distributed locks using Redis (SETNX, Redlock)? Discuss safety and correctness.

Hard · Messaging & In-Memory Data, Redis, Staff

When to use Redis Streams vs Kafka for event streaming?

Easy · Messaging & In-Memory Data, Redis, Fresher

How do you use Redis Pub/Sub? What are its limitations for production messaging?

Hard · Messaging & In-Memory Data, Redis, Staff

Explain Redis transactions (MULTI/EXEC) and optimistic locking with WATCH.

Medium · Messaging & In-Memory Data, Redis, Senior

How do you secure Redis in the cloud (TLS, AUTH, VPC, security groups)?

Medium · Messaging & In-Memory Data, Redis, Senior

How to use Redis as a session store for stateless applications? Discuss consistency and scalability.

Easy · Messaging & In-Memory Data, Redis, Fresher

How do you measure Redis performance and which metrics matter (OPS, latency, memory usage, evictions)?

Medium · Messaging & In-Memory Data, Redis, Senior

How do you shard data manually vs using Redis Cluster?

Medium · Messaging & In-Memory Data, Redis, Senior

Explain memory optimization techniques for Redis (compression, data modeling).

Medium · Messaging & In-Memory Data, Redis, Senior

How do you handle large sorted sets or huge keyspace in Redis effectively?

Medium · Messaging & In-Memory Data, Redis, Senior

How to integrate Redis with AWS: Elasticache vs self-managed Redis on EC2?

Easy · Messaging & In-Memory Data, Redis, Fresher

Describe a production scenario: using Redis for leaderboard and real-time counters—design and pitfalls.

Hard · Messaging & In-Memory Data, Redis, Architect

What are the benefits and trade-offs of asynchronous communication versus synchronous communication?

Hard · Messaging & In-Memory Data, Async Communication, Staff

Describe common async patterns: fire-and-forget, request-reply, event notification, and CQRS.

Easy · Messaging & In-Memory Data, Async Communication, Fresher

How do you design idempotent message handlers for async systems?

Hard · Messaging & In-Memory Data, Async Communication, Staff

Explain callback-based async vs Future/Promise vs reactive streams (Backpressure).

Medium · Messaging & In-Memory Data, Async Communication, Senior

How do Java CompletableFuture and Project Reactor (Flux/Mono) differ? When to use each?

Medium · Messaging & In-Memory Data, Async Communication, Senior

What is reactive programming and the Reactive Streams specification?

Easy · Messaging & In-Memory Data, Async Communication, Fresher

How do you apply backpressure in reactive systems and why is it important?

Medium · Messaging & In-Memory Data, Async Communication, Senior

How do you design end-to-end tracing for async flows (correlation IDs, span propagation)?

Hard · Messaging & In-Memory Data, Async Communication, Staff

What patterns help handle partial failures in async pipelines (retry, circuit breaker, DLQ)?

Easy · Messaging & In-Memory Data, Async Communication, Fresher

How do you implement asynchronous request-reply over Kafka or messaging systems?

Medium · Messaging & In-Memory Data, Async Communication, Senior

What are the challenges of debugging async applications and how to mitigate them?

Easy · Messaging & In-Memory Data, Async Communication, Fresher

How do you guarantee ordering in async systems where ordering matters?

Medium · Messaging & In-Memory Data, Async Communication, Senior

How do you build reactive REST APIs with Spring WebFlux and when to prefer it over Spring MVC?

Medium · Messaging & In-Memory Data, Async Communication, Senior

What are the implications of using non-blocking IO for database access?

Easy · Messaging & In-Memory Data, Async Communication, Fresher

How do you manage thread pools and schedulers in reactive applications?

Medium · Messaging & In-Memory Data, Async Communication, Senior

Explain eventual consistency vs strong consistency in async architectures and trade-offs.

Hard · Messaging & In-Memory Data, Async Communication, Staff

How do you design async workflows that involve multiple microservices and external systems?

Hard · Messaging & In-Memory Data, Async Communication, Staff

How to implement idempotent retries and exponential backoff in asynchronous processing?

Medium · Messaging & In-Memory Data, Async Communication, Senior

How do you coordinate long-running async transactions (Saga, compensation) in practical systems?

Hard · Messaging & In-Memory Data, Async Communication, Staff

What are the best practices for resource cleanup in async/reactive code?

Easy · Messaging & In-Memory Data, Async Communication, Fresher

How do you handle timeouts and cancellation in CompletableFuture and Reactor?

Medium · Messaging & In-Memory Data, Async Communication, Senior

What role do message headers and metadata play in async messaging design?

Hard · Messaging & In-Memory Data, Async Communication, Staff

How do you maintain SLA and observability for async endpoints?

Medium · Messaging & In-Memory Data, Async Communication, Senior

Describe testing strategies for async code (unit tests, integration tests, virtual time).

Easy · Messaging & In-Memory Data, Async Communication, Fresher

How do you migrate blocking services to asynchronous/reactive paradigms—steps, pitfalls, and verification?

Hard · Messaging & In-Memory Data, Async Communication, Architect

What are the core differences between Kafka, RabbitMQ, and AWS SQS?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

When should you pick a message broker vs an event log (Kafka)?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

Explain message durability, persistence, and acknowledgement semantics.

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

How do you design dead-letter queue (DLQ) strategies and poison message handling?

Hard · Messaging & In-Memory Data, Messaging Queues, Staff

How can you ensure message ordering when using queues that do not guarantee it?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

Discuss retry strategies: immediate retry, delayed retry, backoff, and retry count limits.

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

How do you ensure exactly-once processing semantics across queues and downstream systems?

Hard · Messaging & In-Memory Data, Messaging Queues, Staff

What are transactional messaging patterns and how to implement them with JMS/Rabbit/SQS?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

How do you handle large messages (payloads) efficiently with messaging systems?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

How do you design idempotent consumers to tolerate duplicates?

Hard · Messaging & In-Memory Data, Messaging Queues, Staff

Explain consumer scaling patterns (competing consumers, sharding, partitioning).

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

What is message routing and how do exchanges/types work in RabbitMQ?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

How do you handle fanout and multicast messaging patterns?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

How to implement request-reply over message queues?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

What security measures apply to messaging systems (TLS, IAM, ACLs)?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

How to integrate messaging systems with Spring Boot (Spring AMQP, Spring Cloud Stream)?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

What is the role of connectors and bridges (Kafka Connect, SQS-to-Kafka) in hybrid landscapes?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

How to manage schema and payload evolution in message-based systems?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

How do you observe and instrument message throughput, latencies, and queue depth?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

What strategies exist for batching and compression of messages for throughput?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

How to design a transactional outbox pattern to ensure reliable message publishing?

Hard · Messaging & In-Memory Data, Messaging Queues, Staff

How do you manage quotas and throttling for message producers/consumers?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

How to design cross-region messaging and replication for disaster recovery?

Hard · Messaging & In-Memory Data, Messaging Queues, Architect

What are the operational best practices for running managed brokers (MSK, RabbitMQ, SQS)?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

Provide a practical design: order-to-fulfillment flow using SQS + Lambda + DynamoDB—outline components and failure modes.

Hard · Messaging & In-Memory Data, Messaging Queues, Architect

What is a message queue and how does it work?

Easy · Messaging & In-Memory Data, Fundamentals, Fresher

Explain pub/sub vs point-to-point messaging.

Easy · Messaging & In-Memory Data, Fundamentals, Fresher

Why use messaging instead of synchronous REST calls?

Medium · Messaging & In-Memory Data, Fundamentals, Fresher

What is the difference between a message queue and an event log (like Kafka)?

Easy · Messaging & In-Memory Data, Fundamentals, Fresher

What happens when a consumer is down in a message queue system?

Hard · Messaging & In-Memory Data, Fundamentals, Fresher

What is Memcached and when should you use it vs Redis?

Easy · Messaging & In-Memory Data, Caching Patterns, Fresher

How does JVM local caching (Caffeine) complement Redis?

Medium · Messaging & In-Memory Data, Caching Patterns, Senior

How do you invalidate caches using pub/sub or event streams?

Medium · Messaging & In-Memory Data, Caching Patterns, Senior

Design a multi-tier caching strategy (L1 local + L2 Redis + CDN).

Hard · Messaging & In-Memory Data, Caching Patterns, Staff

How does CDC-driven cache warming work?

Hard · Messaging & In-Memory Data, Caching Patterns, Staff

How do you design a globally distributed cache architecture?

Hard · Messaging & In-Memory Data, Caching Patterns, Architect

What is event sourcing and how does it differ from CRUD?

Medium · Messaging & In-Memory Data, Event Architecture, Senior

Explain CQRS and how messaging connects commands and queries.

Hard · Messaging & In-Memory Data, Event Architecture, Staff

Event notification vs event-carried state transfer — when to use each?

Medium · Messaging & In-Memory Data, Event Architecture, Senior

Saga: choreography vs orchestration — explain with examples.

Hard · Messaging & In-Memory Data, Event Architecture, Senior

What are hot partitions in Kafka and how do you fix them?

Medium · Messaging & In-Memory Data, Kafka, Senior

What is KRaft mode in Kafka (without ZooKeeper)?

Medium · Messaging & In-Memory Data, Kafka, Staff

Compare Apache Pulsar vs Kafka.

Medium · Messaging & In-Memory Data, Kafka, Staff

What is tiered storage in Kafka and when is it useful?

Medium · Messaging & In-Memory Data, Kafka, Staff

Redis vs Valkey, Dragonfly, and KeyDB — when to choose each?

Medium · Messaging & In-Memory Data, Redis, Architect

When should you NOT use an in-memory database like Redis?

Medium · Messaging & In-Memory Data, Redis, Architect

How do you handle read-your-writes with Redis read replicas?

Medium · Messaging & In-Memory Data, Redis, Staff

What is split-brain in Redis Sentinel and how do you prevent it?

Hard · Messaging & In-Memory Data, Redis, Staff

What are Redis modules (RedisJSON, RediSearch, RedisBloom)?

Medium · Messaging & In-Memory Data, Redis, Senior

Implement a sliding-window rate limiter with Redis.

Medium · Messaging & In-Memory Data, Redis, Senior

What is Google Cloud Pub/Sub and how does it work?

Easy · Messaging & In-Memory Data, Messaging Queues, Fresher

What is NATS and when would you use it?

Medium · Messaging & In-Memory Data, Messaging Queues, Senior

How do you deduplicate messages using Redis SET NX?

Hard · Messaging & In-Memory Data, Messaging Queues, Staff

How do you design a multi-tenant messaging platform?

Hard · Messaging & In-Memory Data, Platform & Strategy, Architect

What is an event mesh and how do you govern schemas?

Hard · Messaging & In-Memory Data, Platform & Strategy, Architect

How do you handle PII, retention, and GDPR right-to-delete in event systems?

Hard · Messaging & In-Memory Data, Platform & Strategy, Architect

How do you plan capacity for Kafka and Redis in production?

Hard · Messaging & In-Memory Data, Platform & Strategy, Architect

How do you build a self-service messaging platform for engineering teams?

Hard · Messaging & In-Memory Data, Platform & Strategy, Architect

How do you migrate from sync REST to async messaging without big-bang rewrites?

Hard · Messaging & In-Memory Data, Platform & Strategy, Architect