Distributed systems: CAP, quorum, replication, and consensus
System Evolution
Backpressure
When consumers can't keep up, the pressure needs to flow upstream β not explode the queue.
Why the previous solution stopped being enough
Your services can retry safely now. But your order processing pipeline is falling behind. Producers send 10,000 orders/second. Consumers can handle 3,000. The queue is growing. Something is about to break.
Architecture Diagram
Producer A
Producer B
Producer C
Queue
filling up
Consumer
overloaded
Backpressure Signal
Consumer tells producers to slow down
In any producer-consumer system, the producer naturally runs at whatever speed it can. If the consumer is slower, the gap between production rate and consumption rate creates accumulation. Queues fill. Memory grows. Eventually something crashes.
Backpressure is a mechanism where the consumer signals to the producer: 'slow down, I can't keep up.' Instead of the queue growing indefinitely, the producer throttles itself.
The simplest form: if the queue depth exceeds a threshold, the server returns HTTP 429 (Too Many Requests) and the client backs off. This is explicit backpressure β the system communicates its load state.
In streaming systems like Kafka, consumers acknowledge messages and pull at their own pace. The producer doesn't push faster than consumers can acknowledge. This is implicit backpressure built into the protocol.
Without backpressure, overloaded systems tend to cascade β a slow consumer causes queue growth, which causes GC pauses or OOM errors, which causes more failures. With backpressure, the system degrades gracefully: it slows down instead of crashing.
Tradeoffs
Engineering Edge Cases
Backpressure that propagates all the way to the user (e.g., a payment form refusing to submit) needs clear user feedback β not a silent hang.
Persistent backpressure is a scaling signal: your consumer needs more capacity.
What comes next
Your queues are stable now. But your distributed system has many nodes β and you have no idea when one silently dies. You need to know if nodes are alive.
