Module 8: Review & Cheat Sheet

1. Interactive Flashcards

Test your knowledge. Click (or tap) the card to flip it.

2. Cheat Sheet

Messaging Tool Comparison

Feature RabbitMQ (Queue) Kafka (Stream) SQS (Cloud Queue)
Model Push (Smart Broker) Pull (Dumb Broker) Pull
Persistence Transient (Memory) Durable Log (Disk) Durable (Multi-AZ)
Throughput High (~20k/sec) Massive (~2M/sec) High (Unlimited)
Ordering FIFO Per Partition Best-Effort (FIFO optional)
Consumer State Broker tracks Ack Consumer tracks Offset Hidden (Visibility Timeout)
Best For Complex Routing, Jobs CDC, Analytics, Event Sourcing Simple Async Tasks

Reliability Checklist

  1. Idempotency: Use UUID keys to deduplicate processing.
  2. Retries: Use Exponential Backoff + Jitter.
  3. DLQ: Configure a Dead Letter Queue for max retries.
  4. Backpressure: Implement Rate Limiting or Auto-scaling.
  5. Monitoring: Alert on Queue Depth and Consumer Lag.

Kafka Glossary

  • Topic: A category of messages (e.g., logs).
  • Partition: A horizontal shard of a topic. Unit of parallelism.
  • Offset: A unique ID for a message in a partition.
  • Consumer Group: A set of consumers sharing the work of a topic.
  • ISR: In-Sync Replicas. Must be > 1 for high durability.
  • Log Compaction: Retaining only the last known value for each key.

3. Scenario Question: Flash Sale Notifications

Problem: You need to send “Deal Started” push notifications to 10 Million users in 5 minutes. Constraint: Users must receive it fast (Low Latency).

Design:

  1. Pattern: Pub-Sub (Fanout).
  2. Tool: Kafka (High Throughput). RabbitMQ might choke on 10M messages if not sharded.
  3. Optimization:
    • Topic: deals.flash_sale with 100 Partitions.
    • Consumers: 100 Workers (Consumer Group) reading from Kafka.
    • Push Service: Workers call FCM/APNS (Apple/Google Push Service).
  4. Why?: Kafka handles the “Thundering Herd” of 10M events. The workers process at their own pace (Backpressure) without crashing the Push Service.