Distributed systems: CAP, quorum, replication, and consensus
System Evolution
Quorum
Quorum makes read and acknowledged write replica sets overlap; consistency also depends on the protocol.
Why the previous solution stopped being enough
CAP theorem made you realize: you can't just blindly read from any replica. Some replicas might be stale. Reads need to contact replicas that participated in an acknowledged write, plus a protocol for reconciling versions.
Architecture Diagram
Node 1
quorum
Node 2
quorum
Node 3
quorum
Node 4
extra
Node 5
extra
N = 5
total replicas
W = 3
write quorum
R = 3
read quorum
R + W > N β overlap guaranteed
3 + 3 = 6 > 5 β read and acknowledged write sets intersect
Strict majority: βN/2β + 1 Β· N=4 β 3 Β· N=5 β 3 Β· N=6 β 4
A quorum is the minimum number of nodes that must agree for an operation to be considered successful. The magic is in the math: if writes require W nodes to confirm, and reads require R nodes to respond, and R + W > N (total replicas), then reads and writes always overlap by at least one node.
Concrete example with N=5: Set W=3 (wait for 3 write acknowledgements) and R=3 (read from 3 replicas). Since 3 + 3 = 6 > 5, at least 1 replica must be in both sets. A subsequent read can observe an acknowledged write through that shared replica when the protocol reconciles versions correctly.
A strict majority is floor(N/2)+1, written βN/2β+1. For N=4 it is 3, for N=5 it is also 3, and for N=6 it is 4. Two sets that each contain more than half of the same replica group must intersect.
You can tune the tradeoff. W=N, R=1 waits for every write replica but needs only one read response. W=1, R=N waits for one write acknowledgement but every read replica. W=βN/2β+1, R=βN/2β+1 is the balanced majority choice. These counts determine overlap, not a fixed latency or transaction-isolation guarantee.
Cassandra exposes these choices through per-operation consistency levels. QUORUM waits for a strict majority of the replication factor; LOCAL_QUORUM uses a majority within the local datacenter. Matching read/write quorums must cover the same replica group. Cassandra still sends writes to all replicas; W controls how many acknowledgements it waits for.
Apache Cassandra: consistency levels and quorum intersection
Tradeoffs
Engineering Edge Cases
Quorum overlap alone is not linearizability or serializable transactions. Concurrent writes, timestamp ordering and failed writes need protocol-specific handling; Cassandra uses lightweight transactions for conditional updates requiring consensus.
Across multiple datacenters, Cassandra EACH_QUORUM writes require a majority in each datacenter. LOCAL_QUORUM read/write overlap applies within the same datacenter, not to arbitrary reads elsewhere.
What comes next
Quorum handles coordination for reads and writes. But who coordinates the cluster itself? When nodes disagree about state, someone needs to be in charge. That's leader election.
