Kafka vs. RabbitMQ decision guide
A new deep dive lays out when to pick Kafka’s high‑throughput append‑only log versus RabbitMQ’s flexible routing — and warns that wrong queue choices or delivery semantics can cause cascading failures in microservice architectures argued. The piece stresses documenting queue rationale in portfolios, especially for payment and trading systems.
A Confluent benchmark measured Kafka peak throughput at about 605 MB/s and RabbitMQ (with mirrored queues) at about 38 MB/s, while p99 latencies stayed lower for Kafka under high load — numbers that directly quantify the “high‑throughput” vs “low‑latency/low‑volume” tradeoff. confluent.io Kafka added idempotent producers and transactional exactly‑once stream processing in the 0.11 release (announced 2017) via KIP‑98, enabling per‑partition exactly‑once semantics for producers and consumers when configured. confluent.io RabbitMQ exposes publisher confirms and manual consumer acknowledgements (with channel prefetch controls) so publishers can get broker ACKs and consumers can ack only after durable processing, which enforces at‑least‑once delivery semantics unless a strict publisher+consumer protocol is implemented. rabbitmq.com Post‑mortems and cloud guidance repeatedly flag “retry storms” and amplified retries as a root cause of cascading microservice outages, with practical mitigation math like “3 retries × 3 service layers = 27× amplification” and recommendations for exponential backoff, jitter, circuit breakers, and retry budgets. layrs.me Financial workloads require durable audit trails and safe reprocessing: Kafka’s offset/retention model and consumer seek APIs support time‑based replay and forensic reprocessing, while classic RabbitMQ queues remove messages on successful ack (though RabbitMQ Streams is a newer, maturing feature aimed at adding log‑style consumption). allthingsopen.org The story’s call to “document queue rationale” aligns with industry ADR practice: the ADR/MADR templates and AWS Prescriptive Guidance show how to capture decision, alternatives, consequences, authors, and dates to preserve why teams picked Kafka vs RabbitMQ for payment or trading flows. adr.github.io
