πŸ’¬
βœ‰οΈ
πŸ“Έ
▢️

Loading…

Distributed systems: CAP, quorum, replication, and consensus

System Evolution

System Evolution

Step 8 of 14
πŸ“

CAP Theorem

When the network splits, distributed systems must choose: consistent data or staying available.

Why the previous solution stopped being enough

Replication is running smoothly β€” until a network partition splits your cluster in two. Nodes on each side can still accept requests, but they can't communicate with each other. What does each side do?

Architecture Diagram

Consistency

every node returns same data

Availability

always responds

Partition Tolerance

survives network splits

CP systems

HBase, ZooKeeper

AP systems

Cassandra, DynamoDB

CA systems

not recommended

P is non-negotiable in distributed systems. You always choose between C and A.

The CAP Theorem, proven by Eric Brewer, states that a distributed system can only guarantee two of three properties simultaneously: Consistency, Availability, and Partition Tolerance.

Consistency means every read receives the most recent write. If you wrote 'balance = $500', every node immediately reflects $500. Availability means the system always responds β€” even if the response might be stale. Partition Tolerance means the system keeps running even when some network links between nodes fail.

In the real world, network partitions are unavoidable. Cables break, routers crash, cloud availability zones have outages. You cannot build a system that is both consistent and available during a partition β€” you must choose.

CP systems prioritize consistency over availability. During a partition, they refuse to serve reads rather than risk returning stale data. HBase, ZooKeeper, and Redis Cluster (with correct config) are examples. Banking and payment systems typically choose CP.

AP systems prioritize availability over consistency. During a partition, they continue serving requests but may return stale data. DynamoDB, Cassandra, and CouchDB are examples. Social feeds, shopping carts, and DNS are examples where AP is the right tradeoff.

PACELC extends CAP by noting that even without partitions, you trade latency for consistency: synchronous replication is more consistent but slower; async replication is faster but less consistent.

Tradeoffs

CP choiceNever returns stale data. Goes unavailable during partitions. Right for financial systems.
AP choiceAlways responds. May return stale data. Right for social and content systems.
Partition inevitabilityYou cannot design away network partitions. You must design through them.
PACELCIn normal operation (no partition), every system trades latency for consistency.

Engineering Edge Cases

β–Έ

CAP is often oversimplified. Many systems are tunable β€” you can pick consistency levels per operation.

β–Έ

Eventual consistency (AP) systems require application-level conflict resolution when diverged replicas reconcile.

What comes next

You understand the tradeoff. But how do systems actually enforce consistency across replicas when some nodes might be unreachable? That requires a concept called quorum.