A simple blog made with Zola and Duckquill Zola 2026-07-11T00:00:00+00:00 https://cetanu.github.io/tags/sharding/atom.xml 2026-07-11T00:00:00+00:00 2026-07-11T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/how-to-avoid-sharding-yourself/ <p>I watched a video named <a rel="external" href="https://www.youtube.com/watch?v=rxR4nIQ0hCk">The Problem Sharding a Database Solves</a> by <a rel="external" href="https://www.youtube.com/@WebDevCody">Web Dev Cody</a>. He has quite a large number of subscribers, his video popped up on my feed, and I enjoyed the video. Also, the diagramming tool he’s using is cool and if anyone knows what it is - please tell me!</p> <p>However, one thing stuck out to me that I wanted to intercept in order to provide advice on, based on my own real professional experience.</p> <h1 id="the-part-i-m-focusing-on"><a class="zola-anchor" href="#the-part-i-m-focusing-on" aria-label="Anchor link for: the-part-i-m-focusing-on"><i class="icon"></i></a> The part I’m focusing on</h1> <p>It’s not the whole video, it’s just one minor part. Cody delivers on what I think the point of his video is - to describe how sharding solves a particular problem - but there is some detail that comes across as a suggestion for how to implement routing a tenant to the shards and this is what I’ll be discussing.</p> <p>Cody describes a naive approach where you take some kind of tenant identifier and hash it, which gives you a deterministic way to determine which shard the tenant lives on.</p> <p>He then rightly points out that if you pick this approach, as the number and size of tenants grows you’ve now made it harder to rebalance your shards, or migrate tenants, because when you insert new shards, the deterministic hashing essentially breaks.</p> <p>This leads Cody to describe a possibly more elegant approach called “virtual buckets” where instead of a hash being a 1:1 mapping between a tenant and a shard, you take the modulus of the hash which maps a range of tenants to a shard instead.</p> <h1 id="multi-tenant-routing-with-an-algorithm-is-an-anti-pattern"><a class="zola-anchor" href="#multi-tenant-routing-with-an-algorithm-is-an-anti-pattern" aria-label="Anchor link for: multi-tenant-routing-with-an-algorithm-is-an-anti-pattern"><i class="icon"></i></a> Multi-tenant Routing with an Algorithm is an Anti-Pattern</h1> <p>These methods of routing tenants to shards aren’t broken, but let’s call them anti-patterns.</p> <p>First, let’s talk about why hashing is an attractive option.</p> <ul> <li>It’s stateless</li> <li>It’s fast</li> <li>It doesn’t need a database lookup</li> </ul> <p>But it leaves you open to <strong>operational pain</strong>, even with the use of virtual buckets.</p> <h3 id="one-tenant-within-a-bucket-can-dwarf-other-tenants"><a class="zola-anchor" href="#one-tenant-within-a-bucket-can-dwarf-other-tenants" aria-label="Anchor link for: one-tenant-within-a-bucket-can-dwarf-other-tenants"><i class="icon"></i></a> One tenant within a bucket can dwarf other tenants</h3> <p>If we just assume that the Pareto principle will play out eventually, you’re going to have one tenant which consumes 80% of a single shards capacity.</p> <p>In order to rebalance them, you need to change their identifier. Now you have to choose between accepting the noisy-neighbour problem where a huge tenant fights for dominance with the other small tenants on the shard, or you add a special-case to your hashing algorithm that lives on forever, and most likely grows, needing to be maintained by operators.</p> <h3 id="no-flexibility-for-isolation"><a class="zola-anchor" href="#no-flexibility-for-isolation" aria-label="Anchor link for: no-flexibility-for-isolation"><i class="icon"></i></a> No flexibility for isolation</h3> <p>This is semi-related to the above, essentially, if you wanted to have different shards, like a “quarantine” shard for misbehaving tenants, or a “free tier/trial” shard, or an “important enterprise customer” shard with only 1 tenant on it, you again have to special-case the algorithm.</p> <p>These requirements will predictably appear as you grow and get more customers.<br /> Maybe they won’t, but we can take care of this with very little operational friction as we’ll see later.</p> <h3 id="rebalancing-friction-doesn-t-go-away"><a class="zola-anchor" href="#rebalancing-friction-doesn-t-go-away" aria-label="Anchor link for: rebalancing-friction-doesn-t-go-away"><i class="icon"></i></a> Rebalancing friction doesn’t go away</h3> <p>Cody does point out that even when you add new shards, you still might need to rebalance tenants, but you can maybe do this with some background process or lazy migration that writes to both shards at the same time.</p> <p>For some businesses these might be acceptable operational events that someone has to take care of, perhaps after-hours or during scheduled downtime.</p> <p>Note: you <em>can</em> mitigate this particular downside with <a rel="external" href="https://en.wikipedia.org/wiki/Consistent_hashing">Consistent Hashing</a> but the others are still unsolved.</p> <h1 id="your-tenants-deserve-their-own-directory"><a class="zola-anchor" href="#your-tenants-deserve-their-own-directory" aria-label="Anchor link for: your-tenants-deserve-their-own-directory"><i class="icon"></i></a> Your tenants deserve their own directory</h1> <p>I know it’s attractive to use clever software to avoid adding infrastructure to solve a problem, but I think this situation justifies the addition of a database to act as a lookup table or directory for your tenants.</p> <p>For the vast majority of businesses, a dedicated Redis or Postgres instance can serve this purpose and take you very far.</p> <p>Yes, it is a single point of failure, and if it goes down so does your entire app probably, but do a little back of the napkin calculation on the probability of redis (in a cluster, with high-availability), whose only job is to keep key value pairs, becoming suddenly unavailable for any serious amount of time. It probably won’t go under 3 nines of availability.</p> <p>Externalising the tenant routing to a lookup table unlocks advantages.</p> <h3 id="surgical-rebalancing"><a class="zola-anchor" href="#surgical-rebalancing" aria-label="Anchor link for: surgical-rebalancing"><i class="icon"></i></a> Surgical Rebalancing</h3> <p>Your rebalancing goes from a stressful situation of changing the algorithm, propagating it to your gateways and praying, to a 5 step process that an intern can execute without bringing the company down.</p> <ol> <li>Identify: you surface a big noisy tenant via monitoring or regular capacity planning</li> <li>Provision: a new shard is created</li> <li>Sync: background job to copy data, stream new writes in the meantime</li> <li>Switch: single atomic database query to point at the new shard</li> <li>Cleanup: wait a week and delete the old copy</li> </ol> <h3 id="better-isolation"><a class="zola-anchor" href="#better-isolation" aria-label="Anchor link for: better-isolation"><i class="icon"></i></a> Better Isolation</h3> <p>Before, changing the way hashing works to satisfy the needs of a particular tenant or tenants meant that you were changing the algorithm for <em>everyone</em>. This represented a huge blast radius.</p> <p>With a lookup table, you’re changing one record.</p> <h1 id="fives-nines-of-availability-no-i-said-nine-fives"><a class="zola-anchor" href="#fives-nines-of-availability-no-i-said-nine-fives" aria-label="Anchor link for: fives-nines-of-availability-no-i-said-nine-fives"><i class="icon"></i></a> Fives nines of availability? No, I said nine fives…</h1> <p>If you’ve somehow reached the point where that lookup table is no longer sufficient (and you’re still reading this blog? lol), or you just feel uncomfortable with having to make a lookup on every request to a single point of failure, there’s a pattern you can employ to scale things further and bring the probability of an outage to a number approximating zero.</p> <p>You can actually <a rel="external" href="https://www.atlassian.com/blog/atlassian-engineering/aws-scaling-multi-region-low-latency-service">read</a> <a rel="external" href="https://www.atlassian.com/blog/atlassian-engineering/atlassian-critical-services-above-six-nines-of-availability">about</a> how Atlassian solved this exact problem using the <a rel="external" href="https://en.wikipedia.org/wiki/Command_Query_Responsibility_Segregation">CQRS pattern</a>. However, I’m going to explain it here even more concisely.</p> <p>CQRS, or “Command Query Responsibility Segregation”, can enable a caching pattern where distributed nodes are able to keep a local copy of the data that they can query even while the lookup table is offline.</p> <p>Distribute caches to each of the gateways that needs the data, and have those caches update &lt; more detail &gt;</p>