Consistent hashing thread
A detailed social thread breaks down consistent hashing for scaling cache clusters and explains why modulo hashing fails under heavy load spikes. The thread is presented as practical system‑design guidance for scenarios like Black Friday traffic. (x.com)
Consistent hashing solves a scaling problem that breaks simple cache clusters: when servers change, it moves only a small slice of keys instead of almost all of them. (aws.amazon.com) A basic cache sharding scheme often uses a hash of the key and then takes the remainder after dividing by the number of servers, written as `hash(key) % N`. Amazon Web Services calls that “naïve modulo mapping” and warns that adding or removing a node causes widespread key remapping. (aws.amazon.com) In a four-node cluster, a key might land on server 2 because its hash modulo 4 equals 2. If the cluster grows to five nodes, the same key is recalculated modulo 5, and many keys jump to different servers at once, turning cache hits into misses. (aws.amazon.com) Consistent hashing uses a different map: it places both keys and servers on a logical ring and sends each key to the next server clockwise on that ring. Amazon Web Services says that design keeps churn low when buckets change, with only a minimal amount of remapping. (aws.amazon.com) That behavior matters in traffic spikes because cache clusters are often scaled out exactly when demand is surging. Amazon ElastiCache documentation recommends consistent hashing for multi-node Memcached clients so load balancing changes do not force a broad reshuffle of cached data. (aws.amazon.com) The same idea shows up outside caches. NGINX documents a `consistent` hash option for upstream servers and says that when a server is added or removed, only a few keys are remapped, which reduces cache misses. (docs.nginx.com) Memcached’s proxy documentation includes a “basic consistent hashing” configuration for a pool of backend servers. That keeps routing tied to the backend list without recalculating the entire keyspace every time the pool changes. (docs.memcached.org) One practical wrinkle is balance: a plain ring can leave uneven gaps, so many implementations add “virtual nodes,” which place each real server on the ring multiple times. That spreads keys more evenly and reduces the chance that one machine gets a disproportionate share of hot traffic. (ably.com) The older research behind the technique framed the problem in web caches, where hot objects could overwhelm individual machines. A 1997 paper by David Karger and other researchers proposed consistent hashing and random trees as distributed caching protocols for relieving those hot spots. (akamai.com) The practical lesson is simple: modulo hashing is easy to write, but it ties every key to the current server count. Consistent hashing adds more machinery up front so a Black Friday scale-out does not turn the cache layer into a synchronized miss storm. (aws.amazon.com)
