Review & Cheat Sheet

[!IMPORTANT] In this lesson, you will master:

  1. 🧠 Interactive Flashcards: Rapid-fire testing on Fanout, Idempotency, and Zero-Copy mechanics.
  2. 📝 Whiteboard Summary: A 4-quadrant reference for system design interviews, covering core concepts to advanced Kafka trade-offs.
  3. 🕵️ Interview Gauntlet: 50 targeted questions ranging from Apprentice to Architect level.

1. Interactive Flashcards

Test your knowledge. Click (or tap) the card to flip it.

2. Elite Messaging Decision Tree

When designing a system, use this logic to pick your communication hardware pathway.

Need to Decouple Services?
YES (Asynchronous)
One-to-Many?
YES
Pub-Sub / Kafka
NO
Queue (SQS/Rabbit)
NO (Synchronous)
Real-time Streaming?
YES
gRPC / WebSockets
NO
REST API

3. Whiteboard Summary

Use this grid to recall the entire module during an interview.

1. Core Concepts

  • Asynchronous: Fire & Forget.
  • Decoupling: Producers unaware of Consumers.
  • Backpressure: Handling load spikes.
  • Durability: Persisting messages to disk.
  • Event Driven: Reacting to state changes.

2. Patterns

  • Queue (1-to-1): Work distribution.
  • Pub-Sub (1-to-Many): Fanout / Notification.
  • Dead Letter Queue: Handling poison messages.
  • Idempotency Key: Safe retries.
  • Transactional Outbox: Atomic DB + Msg.

3. Kafka Deep Dive

  • Log Abstraction: Append-only file.
  • Partition: Unit of Scalability.
  • Consumer Group: Parallel processing.
  • Zero Copy: sendfile() optimization.
  • ISR: Replication reliability.

4. Trade-offs

  • Push (Low Latency) vs Pull (Control).
  • FIFO vs Partition Ordering.
  • At-Most-Once vs At-Least-Once.
  • Throughput vs Latency (Batching).

4. Interview Gauntlet

Can you survive 50 rapid-fire questions?

Level 1: Apprentice (Basics)

  1. What is the main difference between a Queue and a Topic?
  2. Why do we need message queues in Microservices?
  3. What happens if a consumer crashes while processing a message?
  4. What is a Dead Letter Queue (DLQ)?
  5. What does FIFO stand for?
  6. Explain “Decoupling”.
  7. What is a “Producer”?
  8. What is a “Consumer”?
  9. What is RabbitMQ?
  10. What is Amazon SQS?
  11. Is HTTP synchronous or asynchronous?
  12. What is “Polling”?
  13. What is a “Visibility Timeout”?
  14. Why is tight coupling bad?
  15. What is vertical scaling vs horizontal scaling?

Level 2: Engineer (Patterns)

  1. What is the difference between Push and Pull models?
  2. How do you handle “Thundering Herd” in a Push system?
  3. What is “Fanout”?
  4. What is the difference between a Fanout and a Direct exchange?
  5. Why is Idempotency important in messaging?
  6. How do you implement Idempotency?
  7. What is “At-Least-Once” delivery?
  8. What is “At-Most-Once” delivery?
  9. Why can’t we easily guarantee “Exactly-Once”?
  10. What is a “Race Condition”?
  11. How does Redis help with Idempotency?
  12. What is a “Soft Delete”?
  13. How do you handle message ordering in a scaled system?
  14. What is “Backpressure”?
  15. How does a Circuit Breaker relate to Messaging?

Level 3: Architect (Kafka & Deep Dive)

  1. What is a Kafka Partition?
  2. How does Kafka guarantee ordering?
  3. What is a Consumer Group?
  4. If I have 10 partitions and 20 consumers, what happens?
  5. What is “Zero Copy”?
  6. Explain “Sequential I/O” vs “Random I/O”.
  7. What is an ISR (In-Sync Replica)?
  8. What is acks=all?
  9. How does Kafka support Transactions?
  10. What is Log Compaction?
  11. What is the difference between RabbitMQ and Kafka storage?
  12. How do you reprocess dead letters automatically?
  13. What is “Head of Line Blocking” in queues?
  14. How does Partitioning affect Latency?
  15. What is “Skew” in partitioning?
  16. How do you choose a Partition Key?
  17. What is “Consumer Lag”?
  18. How do you monitor a Kafka cluster?
  19. What is the “Transactional Outbox” pattern?
  20. Design a system to count YouTube views in real-time.
  21. Explain the hardware cost of “At-Least-Once” delivery.

5. Hardware-First Messaging Checklist

Before you deploy your broker, verify these physical limits:

  • Disk IOPS: Is your “Persistent” queue saturating the SSD’s random-write limit?
  • Page Cache Utilization: Is your consumer lag pushing reads out of RAM and onto the Physical Disk (Tail Latency Cliff)?
  • NIC Saturation: Is your Fanout ratio (1 to 100) exceeding the 10Gbps/100Gbps network backplane capacity?
  • Context Switch Overhead: Is your high-concurrency RabbitMQ cluster spending more time in Kernel-mode (interrupts) than processing messages?

6. Staff Engineer Challenge: The “Infinite Retry” Storm

The Scenario: A downstream database is down. You have 1,000,000 messages in your Main Queue.

  • The Bug: Your consumer code has a retry loop with no backoff.
  • The Hardware Impact: The messages are being redelivered at 100k/sec, saturating the Broker’s CPU and disk.

The Question: How would you implement a Hardware-Aware Backoff?

  • Would you use a Retry Queue with TTL?
  • Or a Jittered Exponential Backoff in the consumer?
  • How do you prevent the “Synchronous Disk Flush” from locking up the broker while handling these retries?

7. 🔗 Next Steps