A simple blog made with Zola and Duckquill Zola 2026-07-11T00:00:00+00:00 https://cetanu.github.io/atom.xml 2026-07-11T00:00:00+00:00 2026-07-11T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/how-to-avoid-sharding-yourself/ <p>I watched a video named <a rel="external" href="https://www.youtube.com/watch?v=rxR4nIQ0hCk">The Problem Sharding a Database Solves</a> by <a rel="external" href="https://www.youtube.com/@WebDevCody">Web Dev Cody</a>. He has quite a large number of subscribers, his video popped up on my feed, and I enjoyed the video. Also, the diagramming tool he’s using is cool and if anyone knows what it is - please tell me!</p> <p>However, one thing stuck out to me that I wanted to intercept in order to provide advice on, based on my own real professional experience.</p> <h1 id="the-part-i-m-focusing-on"><a class="zola-anchor" href="#the-part-i-m-focusing-on" aria-label="Anchor link for: the-part-i-m-focusing-on"><i class="icon"></i></a> The part I’m focusing on</h1> <p>It’s not the whole video, it’s just one minor part. Cody delivers on what I think the point of his video is - to describe how sharding solves a particular problem - but there is some detail that comes across as a suggestion for how to implement routing a tenant to the shards and this is what I’ll be discussing.</p> <p>Cody describes a naive approach where you take some kind of tenant identifier and hash it, which gives you a deterministic way to determine which shard the tenant lives on.</p> <p>He then rightly points out that if you pick this approach, as the number and size of tenants grows you’ve now made it harder to rebalance your shards, or migrate tenants, because when you insert new shards, the deterministic hashing essentially breaks.</p> <p>This leads Cody to describe a possibly more elegant approach called “virtual buckets” where instead of a hash being a 1:1 mapping between a tenant and a shard, you take the modulus of the hash which maps a range of tenants to a shard instead.</p> <h1 id="multi-tenant-routing-with-an-algorithm-is-an-anti-pattern"><a class="zola-anchor" href="#multi-tenant-routing-with-an-algorithm-is-an-anti-pattern" aria-label="Anchor link for: multi-tenant-routing-with-an-algorithm-is-an-anti-pattern"><i class="icon"></i></a> Multi-tenant Routing with an Algorithm is an Anti-Pattern</h1> <p>These methods of routing tenants to shards aren’t broken, but let’s call them anti-patterns.</p> <p>First, let’s talk about why hashing is an attractive option.</p> <ul> <li>It’s stateless</li> <li>It’s fast</li> <li>It doesn’t need a database lookup</li> </ul> <p>But it leaves you open to <strong>operational pain</strong>, even with the use of virtual buckets.</p> <h3 id="one-tenant-within-a-bucket-can-dwarf-other-tenants"><a class="zola-anchor" href="#one-tenant-within-a-bucket-can-dwarf-other-tenants" aria-label="Anchor link for: one-tenant-within-a-bucket-can-dwarf-other-tenants"><i class="icon"></i></a> One tenant within a bucket can dwarf other tenants</h3> <p>If we just assume that the Pareto principle will play out eventually, you’re going to have one tenant which consumes 80% of a single shards capacity.</p> <p>In order to rebalance them, you need to change their identifier. Now you have to choose between accepting the noisy-neighbour problem where a huge tenant fights for dominance with the other small tenants on the shard, or you add a special-case to your hashing algorithm that lives on forever, and most likely grows, needing to be maintained by operators.</p> <h3 id="no-flexibility-for-isolation"><a class="zola-anchor" href="#no-flexibility-for-isolation" aria-label="Anchor link for: no-flexibility-for-isolation"><i class="icon"></i></a> No flexibility for isolation</h3> <p>This is semi-related to the above, essentially, if you wanted to have different shards, like a “quarantine” shard for misbehaving tenants, or a “free tier/trial” shard, or an “important enterprise customer” shard with only 1 tenant on it, you again have to special-case the algorithm.</p> <p>These requirements will predictably appear as you grow and get more customers.<br /> Maybe they won’t, but we can take care of this with very little operational friction as we’ll see later.</p> <h3 id="rebalancing-friction-doesn-t-go-away"><a class="zola-anchor" href="#rebalancing-friction-doesn-t-go-away" aria-label="Anchor link for: rebalancing-friction-doesn-t-go-away"><i class="icon"></i></a> Rebalancing friction doesn’t go away</h3> <p>Cody does point out that even when you add new shards, you still might need to rebalance tenants, but you can maybe do this with some background process or lazy migration that writes to both shards at the same time.</p> <p>For some businesses these might be acceptable operational events that someone has to take care of, perhaps after-hours or during scheduled downtime.</p> <p>Note: you <em>can</em> mitigate this particular downside with <a rel="external" href="https://en.wikipedia.org/wiki/Consistent_hashing">Consistent Hashing</a> but the others are still unsolved.</p> <h1 id="your-tenants-deserve-their-own-directory"><a class="zola-anchor" href="#your-tenants-deserve-their-own-directory" aria-label="Anchor link for: your-tenants-deserve-their-own-directory"><i class="icon"></i></a> Your tenants deserve their own directory</h1> <p>I know it’s attractive to use clever software to avoid adding infrastructure to solve a problem, but I think this situation justifies the addition of a database to act as a lookup table or directory for your tenants.</p> <p>For the vast majority of businesses, a dedicated Redis or Postgres instance can serve this purpose and take you very far.</p> <p>Yes, it is a single point of failure, and if it goes down so does your entire app probably, but do a little back of the napkin calculation on the probability of redis (in a cluster, with high-availability), whose only job is to keep key value pairs, becoming suddenly unavailable for any serious amount of time. It probably won’t go under 3 nines of availability.</p> <p>Externalising the tenant routing to a lookup table unlocks advantages.</p> <h3 id="surgical-rebalancing"><a class="zola-anchor" href="#surgical-rebalancing" aria-label="Anchor link for: surgical-rebalancing"><i class="icon"></i></a> Surgical Rebalancing</h3> <p>Your rebalancing goes from a stressful situation of changing the algorithm, propagating it to your gateways and praying, to a 5 step process that an intern can execute without bringing the company down.</p> <ol> <li>Identify: you surface a big noisy tenant via monitoring or regular capacity planning</li> <li>Provision: a new shard is created</li> <li>Sync: background job to copy data, stream new writes in the meantime</li> <li>Switch: single atomic database query to point at the new shard</li> <li>Cleanup: wait a week and delete the old copy</li> </ol> <h3 id="better-isolation"><a class="zola-anchor" href="#better-isolation" aria-label="Anchor link for: better-isolation"><i class="icon"></i></a> Better Isolation</h3> <p>Before, changing the way hashing works to satisfy the needs of a particular tenant or tenants meant that you were changing the algorithm for <em>everyone</em>. This represented a huge blast radius.</p> <p>With a lookup table, you’re changing one record.</p> <h1 id="fives-nines-of-availability-no-i-said-nine-fives"><a class="zola-anchor" href="#fives-nines-of-availability-no-i-said-nine-fives" aria-label="Anchor link for: fives-nines-of-availability-no-i-said-nine-fives"><i class="icon"></i></a> Fives nines of availability? No, I said nine fives…</h1> <p>If you’ve somehow reached the point where that lookup table is no longer sufficient (and you’re still reading this blog? lol), or you just feel uncomfortable with having to make a lookup on every request to a single point of failure, there’s a pattern you can employ to scale things further and bring the probability of an outage to a number approximating zero.</p> <p>You can actually <a rel="external" href="https://www.atlassian.com/blog/atlassian-engineering/aws-scaling-multi-region-low-latency-service">read</a> <a rel="external" href="https://www.atlassian.com/blog/atlassian-engineering/atlassian-critical-services-above-six-nines-of-availability">about</a> how Atlassian solved this exact problem using the <a rel="external" href="https://en.wikipedia.org/wiki/Command_Query_Responsibility_Segregation">CQRS pattern</a>. However, I’m going to explain it here even more concisely.</p> <p>CQRS, or “Command Query Responsibility Segregation”, can enable a caching pattern where distributed nodes are able to keep a local copy of the data that they can query even while the lookup table is offline.</p> <p>Distribute caches to each of the gateways that needs the data, and have those caches update &lt; more detail &gt;</p> 2026-07-10T00:00:00+00:00 2026-07-10T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/do-we-really-need-tls-for-cat-pictures/ <p>It is a common piece of skepticism: <em>"Is encrypting every single corner of the web actually necessary? If I’m just reading a static blog post or looking at a cute picture of a cat, there are no passwords or credit cards involved. Why do we need to wrap all of that in TLS? Aren’t we just wasting a massive amount of CPU cycles globally?"</em></p> <p>On the surface, it seems like a reasonable complaint. Wrapping every HTTP response in a layer of cryptography must have a cost.</p> <p>But when you dig into the mechanics of modern networks and the history of how ISPs behave when we leave our traffic in plaintext, the answer becomes obvious. Ubiquitous TLS isn’t paranoia or wasteful bloat—it is the bare minimum required to keep the web functional.</p> <p>Let’s break down the actual compute cost of TLS and look at what happens to your data when you don’t encrypt it.</p> <hr /> <h2 id="the-compute-myth-is-tls-actually-wasting-cpu"><a class="zola-anchor" href="#the-compute-myth-is-tls-actually-wasting-cpu" aria-label="Anchor link for: the-compute-myth-is-tls-actually-wasting-cpu"><i class="icon"></i></a> The Compute Myth: Is TLS Actually Wasting CPU?</h2> <p>Historically, yes, SSL/TLS was computationally expensive. In the 1990s and early 2000s, setting up a secure connection and encrypting a stream of data was a heavy burden for servers. Websites that wanted encryption often had to buy dedicated, expensive SSL accelerator expansion cards to offload the cryptographic math from the main CPU.</p> <p>Today, that bottleneck is long gone.</p> <h3 id="1-hardware-accelerated-cryptography-aes-ni-modern-cpus-both-client-side"><a class="zola-anchor" href="#1-hardware-accelerated-cryptography-aes-ni-modern-cpus-both-client-side" aria-label="Anchor link for: 1-hardware-accelerated-cryptography-aes-ni-modern-cpus-both-client-side"><i class="icon"></i></a> 1. Hardware-Accelerated Cryptography (AES-NI) Modern CPUs (both client-side</h3> <p>and server-side) have dedicated hardware instructions built directly into the silicon to handle symmetric encryption. The most famous of these is Intel and AMD’s <strong>AES-NI</strong> (Advanced Encryption Standard New Instructions).</p> <p>With AES-NI, the CPU doesn’t need to run a software loop to encrypt data block-by-block. Instead, it offloads it to hardware execution units that run the operations in a handful of clock cycles.</p> <p>Because of this, modern processors can encrypt and decrypt AES-GCM (the primary cipher suite used in TLS 1.3) at near-memory speeds. A single modern CPU core can easily encrypt multiple gigabits of data per second. On a typical web server, the CPU overhead of encrypting the payload is a fraction of a percent.</p> <h3 id="2-tls-1-3-handshake-optimizations-the-most-computationally-intensive-part"><a class="zola-anchor" href="#2-tls-1-3-handshake-optimizations-the-most-computationally-intensive-part" aria-label="Anchor link for: 2-tls-1-3-handshake-optimizations-the-most-computationally-intensive-part"><i class="icon"></i></a> 2. TLS 1.3 Handshake Optimizations The most computationally intensive part</h3> <p>of TLS isn’t actually encrypting the data stream (symmetric cryptography)—it is the initial handshake where the client and server negotiate keys (asymmetric cryptography).</p> <p>TLS 1.3, finalized in 2018, radically streamlined this process:</p> <ul> <li><strong>One Round-Trip (1-RTT):</strong> The handshake was cut from two round-trips to one, reducing latency.</li> <li><strong>Pre-Shared Key (PSK) Session Resumption:</strong> Returning visitors can resume a session in zero round-trips (0-RTT), skipping the expensive asymmetric key generation entirely.</li> <li><strong>Faster Algorithms:</strong> Modern deployments favor elliptic curve algorithms like <code>X25519</code> for key exchange and <code>ECDSA</code> or <code>Ed25519</code> for signatures. These are orders of magnitude faster and lighter than the classic, bulky RSA keys of the past.</li> </ul> <p>If you profile a modern web application, the resources spent on TLS are a drop in the bucket. Parsing a JSON payload, running a database query, rendering a React component, or executing a snippet of JavaScript takes orders of magnitude more CPU cycles than encrypting a packet.</p> <hr /> <h2 id="if-the-compute-cost-is-low-why-encrypt-cat-pictures"><a class="zola-anchor" href="#if-the-compute-cost-is-low-why-encrypt-cat-pictures" aria-label="Anchor link for: if-the-compute-cost-is-low-why-encrypt-cat-pictures"><i class="icon"></i></a> If the Compute Cost is Low, Why Encrypt Cat Pictures?</h2> <p>If the computational cost is negligible, we still have to ask: what is the point of encrypting non-sensitive content? If someone intercepts your request to see a cat picture, they just see a cat picture. Who cares?</p> <p>The issue isn’t just <strong>confidentiality</strong> (keeping secrets secret). The real threat is <strong>integrity</strong> (making sure the data isn’t tampered with).</p> <p>Without TLS, your connection is just plain HTTP. That means every router, switch, and intermediate network node between your browser and the web server can read your traffic—and more importantly, they can <strong>modify</strong> it.</p> <p>This isn’t a theoretical threat. Throughout the 2010s, Internet Service Providers (ISPs), public Wi-Fi operators, and network gateways repeatedly behaved like active man-in-the-middle attackers to monetize and track plaintext web traffic.</p> <p>Here are a few of the most egregious examples of what ISPs did when they were allowed to read plaintext streams.</p> <hr /> <h2 id="when-isps-act-like-malicious-actors"><a class="zola-anchor" href="#when-isps-act-like-malicious-actors" aria-label="Anchor link for: when-isps-act-like-malicious-actors"><i class="icon"></i></a> When ISPs Act Like Malicious Actors</h2> <h3 id="1-verizon-s-supercookie-uidh"><a class="zola-anchor" href="#1-verizon-s-supercookie-uidh" aria-label="Anchor link for: 1-verizon-s-supercookie-uidh"><i class="icon"></i></a> 1. Verizon’s “Supercookie” (UIDH)</h3> <p>From 2012 to 2016, Verizon Wireless silently injected a unique tracking header called the <strong>UIDH</strong> (Unique Identifier Header) into every unencrypted HTTP request sent by its mobile users.</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="http"><span class="giallo-l"><span style="color: #BB9AF7;">GET</span><span> /cat-pictures</span><span style="color: #89DDFF;"> HTTP</span><span>/</span><span style="color: #FF9E64;">1.1</span></span> <span class="giallo-l"><span style="color: #F7768E;">Host</span><span style="color: #89DDFF;">:</span><span style="color: #9ECE6A;"> example.com</span></span> <span class="giallo-l"><span style="color: #F7768E;">X-UIDH</span><span style="color: #89DDFF;">:</span><span style="color: #9ECE6A;"> 9f8a3c2e1b0d7f... &lt;-- Injected by Verizon&#39;s network</span></span></code></pre> <p>Because HTTP traffic was plaintext, Verizon’s network equipment could intercept the packets in transit, append this custom header, and forward it to the destination server.</p> <p>This header acted as an undeletable tracking cookie. Even if users cleared their browser cookies, blocked trackers, or used incognito mode, third-party advertising networks (like Turn) could read this injected <code>X-UIDH</code> header and use it to reconstruct the user’s tracking profile across the web.</p> <p>Because it was done at the network level, users had no way to disable it. It was only stopped after a security researcher exposed the practice, leading to public outcry and a <strong>$1.35 million fine</strong> from the FCC in 2016. If the sites had been using HTTPS, Verizon would not have been able to inject the header without breaking the cryptographic signature of the packet.</p> <h3 id="2-comcast-s-javascript-injection-in-2017-comcast-users-began-noticing"><a class="zola-anchor" href="#2-comcast-s-javascript-injection-in-2017-comcast-users-began-noticing" aria-label="Anchor link for: 2-comcast-s-javascript-injection-in-2017-comcast-users-began-noticing"><i class="icon"></i></a> 2. Comcast’s JavaScript Injection In 2017, Comcast users began noticing</h3> <p>strange popups appearing on top of the websites they were browsing.</p> <p>Comcast was using deep packet inspection (DPI) to monitor customer traffic. When they detected a user was browsing an unencrypted HTTP site, they would modify the HTML on the fly, injecting custom JavaScript code into the webpage before it reached the user’s browser.</p> <p>The injected code was used to display “courtesy notices” telling users they were approaching their monthly data cap, or warning them that their modem was out of date.</p> <p>While Comcast claimed this was a feature to help consumers, injecting arbitrary JavaScript into third-party websites is incredibly dangerous. It can break the site’s layout, cause conflicts with the site’s own scripts, and introduce major security vulnerabilities. If a hacker managed to compromise Comcast’s injection system, they could have executed arbitrary code on millions of customer browsers.</p> <h3 id="3-hotel-and-airport-wi-fi-ad-injection-if-you-ve-ever-connected-to-a-hotel"><a class="zola-anchor" href="#3-hotel-and-airport-wi-fi-ad-injection-if-you-ve-ever-connected-to-a-hotel" aria-label="Anchor link for: 3-hotel-and-airport-wi-fi-ad-injection-if-you-ve-ever-connected-to-a-hotel"><i class="icon"></i></a> 3. Hotel and Airport Wi-Fi Ad Injection If you’ve ever connected to a hotel</h3> <p>or airport Wi-Fi network and seen a strange floating toolbar at the bottom of every page, or unexpected ads on sites that don’t usually host them, you’ve experienced ad injection.</p> <p>Public Wi-Fi gateways frequently intercept HTTP traffic to inject local advertisements, promotions, or terms-of-service agreements. By rewriting the HTML of the websites you visit, they degrade your browsing experience and can expose your browser to malicious ads (malvertising).</p> <h3 id="4-dns-hijacking-and-search-redirection-many-isps-including-charter-cox"><a class="zola-anchor" href="#4-dns-hijacking-and-search-redirection-many-isps-including-charter-cox" aria-label="Anchor link for: 4-dns-hijacking-and-search-redirection-many-isps-including-charter-cox"><i class="icon"></i></a> 4. DNS Hijacking and Search Redirection Many ISPs (including Charter, Cox,</h3> <p>and Rogers) historically practiced DNS hijacking on unencrypted traffic. If you typed in a non-existent URL (like <code>http://this-does-not-exist.com</code>), instead of showing a browser error page, the ISP would intercept the request and redirect you to a custom search page loaded with ads and sponsored links, monetizing your typos.</p> <hr /> <h2 id="the-three-pillars-of-tls"><a class="zola-anchor" href="#the-three-pillars-of-tls" aria-label="Anchor link for: the-three-pillars-of-tls"><i class="icon"></i></a> The Three Pillars of TLS</h2> <p>When we encrypt the web, we aren’t just hiding passwords. We are establishing three fundamental guarantees:</p> <table><thead><tr><th style="text-align: left">Guarantee</th><th style="text-align: left">What It Means</th><th style="text-align: left">What Happens Without It</th></tr></thead><tbody> <tr><td style="text-align: left"><strong>Confidentiality</strong></td><td style="text-align: left">Only you and the server can read the data.</td><td style="text-align: left">ISPs, governments, or hackers on your local Wi-Fi can see exactly which pages, articles, and pictures you are viewing.</td></tr> <tr><td style="text-align: left"><strong>Integrity</strong></td><td style="text-align: left">The data cannot be modified in transit.</td><td style="text-align: left">Network middleboxes can inject ads, tracking headers, popups, or malicious scripts into the page.</td></tr> <tr><td style="text-align: left"><strong>Authenticity</strong></td><td style="text-align: left">You are communicating with the real website.</td><td style="text-align: left">An attacker can spoof the website’s identity (e.g., DNS poisoning) and serve you a fake page without your browser raising any warnings.</td></tr> </tbody></table> <p>Even if you are only looking at cat pictures, you deserve to know that the cat pictures you are seeing are the actual ones sent by the server, and not a modified payload injected with tracking scripts and banner ads.</p> <h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="icon"></i></a> Conclusion</h2> <p>The push for a 100% encrypted web—championed by projects like <em>Let’s Encrypt</em>, search engines prioritizing HTTPS, and browsers displaying warnings for insecure sites—was a necessary response to the reality of the internet’s infrastructure.</p> <p>The internet is not a direct wire between your computer and a server; it is a chain of intermediaries. And history has proven that if those intermediaries are given the power to read and modify your traffic, they will exploit it for profit.</p> <p>Thanks to modern hardware acceleration like AES-NI and the design of TLS 1.3, we can secure the entire web with virtually zero performance penalty. Encrypted cat pictures aren’t a waste of compute—they are a victory for user privacy and web integrity.</p> 2026-07-07T00:00:00+00:00 2026-07-07T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/false-sharing/ <p>In previous blogs I talked a little bit about <a href="https://cetanu.github.io/blog/why-numa-is-important-in-trading/">NUMA</a> and the <a href="https://cetanu.github.io/blog/translation-lookaside-buffer/">TLB</a>, if you haven’t read those, I recommend starting there before reading this.</p> <p>You should already understand that accessing main memory is relatively slow for the CPU, especially in a low-latency environment, and you may be familiar with the fact that the CPU has small caches of varying sizes on the chip, known as the L1, L2, and L3 caches.</p> <p>As their name implies, there are levels, or a hierarchy, from fastest (and smallest) to slowest (and largest). The L1 cache is also typically split into two caches, one being an instruction cache and the other being a data cache.</p> <p>Fun fact: these caches are usually made of SRAM and usually they are the largest part on the chip.</p> <img class="no-hover"alt="relative speed of hardware"src="&#x2F;img&#x2F;relative-hw-speed.png"/><h2 id="requesting-data-from-the-cache"><a class="zola-anchor" href="#requesting-data-from-the-cache" aria-label="Anchor link for: requesting-data-from-the-cache"><i class="icon"></i></a> Requesting data from the cache</h2> <p>You may have heard of “cache lines” before (also called cache blocks), maybe on some article about performance on hackernews.</p> <p>A cache line can be thought of a single unit of data which is transferred from RAM to the cache. Usually one cache line is 64 bytes, but this can differ depending on your CPU architecture.</p> <p>This introduces the concept of cache locality. When the CPU needs to access the same memory several times in a row, after it’s already been pulled into the L1 cache, this is referred to as <strong>temporal locality</strong>. When the CPU pulls in a cache line and all the data it needs is within that block, it’s referred to as spatial locality. A good example of <strong>spatial locality</strong> is when you have an array that fits into one cache line, resulting in traversal being extremely fast. An example of potentially terrible cache performance might be something like a linked-list.</p> <p>On systems with multiple CPU cores, they might share the same chunk of memory in their local caches. Most processors use what is known as the MESI protocol to prevent overwriting each others data. Jon Gjengset has a <a href="https://cetanu.github.io/blog/false-sharing/www.youtube.com/watch?v=tND-wBBZ8RY">great talk</a> about Mutexes which also covers the MESI protocol.</p> <h2 id="where-it-can-go-wrong"><a class="zola-anchor" href="#where-it-can-go-wrong" aria-label="Anchor link for: where-it-can-go-wrong"><i class="icon"></i></a> Where it can go wrong</h2> <p>CPUs can enter what is called a coherence livelock or cache-line pingponging, where multiple cores fight over the same memory, or the same cache line.</p> <p>The first way this can happen is called <strong>False Sharing</strong>, where your data may reside in a cache line, but it may not be occupying that entire cache line by itself. If your data sits in the first half but some other data sits in the latter half, it’s possible that two CPUs may continuously write to the same cache line, requiring them to communicate with each other to gain ownership in order to modify the cache line. If you watched Jon’s video above you would know that this can eat up dozens of precious nanoseconds, just for the comms between CPUs. As you increase the number of cores on a system this problem can be exacerbated.</p> <img class="no-hover"alt="diagram of false sharing"src="&#x2F;img&#x2F;false-sharing.png"/> <p>The second way is <strong>True Sharing</strong>, where cores may write to the same variable or data structure in memory. This could happen during atomic operations or when using synchronization primitives such as mutexes, semaphores, barriers, spinlocks, and so on.</p> <h2 id="how-to-protect-against-it"><a class="zola-anchor" href="#how-to-protect-against-it" aria-label="Anchor link for: how-to-protect-against-it"><i class="icon"></i></a> How to protect against it</h2> <p>At least for false sharing, the solution seems simple: just pad your data structure or variable so that it occupies an entire cache line. Obviously it would be silly to do this with every variable in your program, so you have to pick the time-critical parts of your application here.</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">use</span><span style="color: #0DB9D7;"> std</span><span style="color: #89DDFF;">::</span><span style="color: #0DB9D7;">sync</span><span style="color: #89DDFF;">::</span><span style="color: #0DB9D7;">atomic</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AtomicU64</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #89DDFF;">#[</span><span>repr</span><span style="color: #89DDFF;">(</span><span>align</span><span style="color: #89DDFF;">(</span><span>64</span><span style="color: #89DDFF;">))]</span></span> <span class="giallo-l"><span style="color: #89DDFF;">pub</span><span style="color: #BB9AF7;"> struct</span><span style="color: #C0CAF5;"> PaddedAtomic</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> pub</span><span style="color: #C0CAF5;"> value</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> AtomicU64</span><span style="color: #89DDFF;">,</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre> <p>For true sharing, it seems quite a bit more complicated to get right. Sometimes you really do need atomic operations on global memory.</p> <p>You could pull data into thread local storage, only merging back at the end of expensive or time critical execution.</p> <p>You could reorganize your program to do batching to reduce how often it needs to write back to the shared memory.</p> <p>Maybe a lock-free data structure could be used.</p> <p>The solution most likely depends on exactly what you are willing to accept in terms of trade-offs.</p> <h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="icon"></i></a> Conclusion</h2> <p>Like I usually say, for the most part things like this are unnoticeable and work just fine for the majority of situations. If you’re running a flask webapp you probably don’t and should not care about this.</p> <p>When you require low-latency, then you need to understand how to write your programs such that they exploit cache-lines for your architecture because it makes the difference between a 1ns L1 cache hit, or falling off a cliff with a 100x slower execution.</p> 2026-06-30T00:00:00+00:00 2026-06-30T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/i-ignored-kubernetes-for-12-years/ <h1 id="my-introduction-to-kubernetes"><a class="zola-anchor" href="#my-introduction-to-kubernetes" aria-label="Anchor link for: my-introduction-to-kubernetes"><i class="icon"></i></a> My introduction to Kubernetes</h1> <p>Like many people I heard about kubernetes because it was software from Google that people were hyped about. I understood a little bit about it - it’s a container orchestrator, right?</p> <p>I also heard about painful experiences that people had with it and I got this impression that it was way too complicated for what it provided in most cases.</p> <p>We’ve all heard the meme “I deployed my blog on kubernetes” as a classic example of overengineering, and resume driven development when a team sets up kubernetes for a single application.</p> <p>While working in the industry, I was happy to let other people deal with this perceived complexity. I was willing to let them work out all the problems and figure out how to offer kubernetes to me as an end-user.</p> <h1 id="post-layoff"><a class="zola-anchor" href="#post-layoff" aria-label="Anchor link for: post-layoff"><i class="icon"></i></a> Post-layoff</h1> <p>However, as time went on, it seemed that the entire industry made a decision to adopt kubernetes, even if the overheads and complexity were of some cost, because the cost was outweighed by standardisation.</p> <p>Suddenly, every SRE / platform engineer role seemed to require knowledge of kubernetes. It was never too clear how much knowledge or skill was required, and there have been several interviews in which I’ve expressed that I don’t have any experience operating kubernetes (probably a bad thing to say in an interview) and yet interviewers did not seem to be able to give me a picture of how they use kubernetes and what I would need to know in order to sufficiently operate their clusters/infrastructure.</p> <h1 id="start-of-the-learning-journey"><a class="zola-anchor" href="#start-of-the-learning-journey" aria-label="Anchor link for: start-of-the-learning-journey"><i class="icon"></i></a> Start of the learning journey</h1> <p>So after being laid off in March 2026, it became apparent that learning kubernetes would be a good idea. Later in May, when my video went viral and an audience appeared with the desire to learn new things, particularly by being taught by myself, it made even more sense to learn kubernetes because it would give me the opportunity to both learn and teach at the same time.</p> <p>So I embarked on a journey to learn kubernetes with the least amount of patience possible, pressuring myself to learn it in a week. Over 7 live-streams, approximately 2-3 hours each, I gathered a decent picture of the kubernetes landscape, a bit of history, and what tools people use to deploy and manage kubernetes in modern times.</p> <h2 id=""><a class="zola-anchor" href="#" aria-label="Anchor link for: "><i class="icon"></i></a> </h2> <p>The very first thing I did was started streaming on Twitch, with the stream title “Added Kubernetes to my CV, now I have to learn it” and opened a browser, went to the kubernetes website and started reading docs linearly. After going through various concepts and basically trying to memorize them and failing, a few chatters informed me of Kelsey Hightower’s guide “Learning Kubernetes The Hard Way”.</p> <p>I started going through this tutorial, and found that it required at least 3 machines with 2GB of memory each, and I didn’t have that hardware laying around, and I didn’t want to run them on my stream PC. So I signed up to AWS and started to create EC2 instances manually, and SSHing onto them to set up hostnames and run OpenSSL commands to do PKI.</p> <p>At some point the tutorial was telling me to execute blobs of shell scripts and I became enraged at the suggestion because it’s been two decades since the advent of configuration management, and at least a decade since kubernetes has been released. So I thought that all this friction had already been smoothed out.</p> <p>In hindsight, the entire purpose of Kelsey’s guide is to expose someone to these low-level concerns. However, I am personally more in favor of getting something practical up and running quickly, and dealing with problems as they arise.</p> <p>So I started an argument with Gemini about Kelsey’s guide, and it graciously agreed that no sane engineer would set up kubernetes by following that guide in a profession scenario. It led me to discover <code>kubeadm</code> which gave me a way to bootstrap the cluster nodes in one command.</p> <p>I used this newly found tool to then set all the infrastructure up with Cloudformation, running the tool during the init of the EC2 instances via UserData. I successfully got a cluster running and then tried to use Helm to install “charts” to get a valheim server pod running.</p> <p>Using helm got a bit messy, and a bunch of pods were crashing, and to be honest I had no idea why. My hypothesis at the time was that the process of pushing YAML to kube via helm and kubectl was fragile and that it was running into issues with state becoming corrupted or partially deployed.</p> <p>Finding myself frustrated, I took a break to think about what my next approach would be.</p> <p>I had a friend tell me about his setup where he doesn’t run any helm commands against the cluster, but instead uses it to produce kubeconfigs as build artifacts which are then ingested via GitOps.</p> <p>I also started to remember a recommendation to use Talos Linux from another peer so I decided to see what that was about. It sold this idea of an immutable kubernetes cluster which supposedly would prevent the kind of state corruption that I thought I was running into.</p> <p>At this stage I decided to completely pivot, use Talos Linux, and switch to Pulumi instead of Cloudformation for the infrastructure even though it would cost me some time in the learning process.</p> <p>I quite enjoyed Pulumi; it feels like a type-checked infrastructure, which is cool. I used the golang SDK.</p> <p>After a bunch of troubleshooting and looking at logs of pods, and feeding those logs into Gemini, I discovered that there was a bootstrap dependency issue where other pods like Argo required that AWS CCM was setup before it would work properly.</p> <p>After this, it was pretty smooth sailing. I could add CRDs and charts to my gitrepo and argo would pick them up and provision pods quickly. I started to thinkabout how to platformatize everything that I’d built but I realised that thiscan become quite subjective, and so for this article I won’t go into that atall, but the options that I was aware of were tools like KubeVela, Crossplane,and Backstage.</p> <blockquote class="warning"> <p class="alert-title"> <i class="icon"></i>Warning</p> <p>Still writing this up</p> </blockquote> <iframe class="youtube-embed" src="https://www.youtube-nocookie.com/embed/V8iT_2YHtns" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen> </iframe> <!-- < here is where I boil the entire article to its essence > Two paths: ## Managed-k8s * No-brainer * Good for small scale * EKS - Locked into AWS Bottlerocket * GKE - ??? * AKS - ??? ## Run-Your-Own-Cluster * Don't want vendor lock-in * Strong need for multi-cloud, especially mixed with baremetal * --- # notes * looked up docs on kubernetes.org * kubernetes the hard way - manually provisioning kube nodes with AWS - hated it, wanted something more polished - started asking LLMs about better ways * discovered kubeadm - started using cloudformation to provision the nodes - created token on the control-plane via userdata script - uploaded the token to AWS ParameterStore - worker nodes pulled the token and joined cluster - tried using Helm to provision pods and such * discovered Talos linux - Replaced the EC2s (Ubuntu) with Talos - Rewrote the Cloudformation into Pulumi - Used a Pulumi library to do the Talos bootstrapping - Looots of troubleshooting - Found that AWS CCM was a dependency of Argo - Fixed the bootstrap, Argo working - Started deploying pods - Logged into Valheim game server as a test - Made a cats server * Started thinking about how to really platformatize kube - found KubeVela - learnt about the OAM (Open Application Model) - refactored existing pods into OAM apps - we are now here --- Of course, it's obvious that people do this usually as an exercise to learn kubernetes in whatever way they can, as did I with my recent adventure to deploy a single valheim server. At a kubernetes interview they may ask you about Operators and CRDs. Also there is a question like "What exactly happens when you run kubectl apply -f deployment.yaml"? kubestronaut short: what is the best piece of advice you've ever gotten? video: recent experience with interviews * coding portion - fundamental knowledge of DSA - methodical approach: * making a naive version first * addressing edge cases * optimizing performance * awareness of time/space complexity * communication the entire way through - justifying tradeoffs * system design - requirements gathering to a professional degree * understand what the hiring company wants/needs --> 2026-04-28T00:00:00+00:00 2026-04-28T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/translation-lookaside-buffer/ <h2 id="the-illusion-of-memory"><a class="zola-anchor" href="#the-illusion-of-memory" aria-label="Anchor link for: the-illusion-of-memory"><i class="icon"></i></a> The Illusion of Memory</h2> <p>Applications do not allocate memory directly to physical addresses on the RAM. They are presented with virtual memory by the operating system, so that it can pretend that the application has a contiguous block of memory when in reality its memory is scattered across the physical RAM in fixed-size chunks of memory called pages, kept track of in a page table.</p> <p>When a process starts, the operating system allocates a range of virtual addresses to it in the form of pages, but leaves the entries in the page table blank until they’re accessed (i.e. they are lazy-loaded)</p> <p>If the process tries to access memory, the operating system follows the virtual memory address to the physical memory address and hands back the data. The actual translation from virtual address to physical address is not performed by the OS but by a dedicated hardware component on the CPU called a memory management unit (MMU).</p> <p>Sometimes, if RAM is running low or hasn’t been accessed in a while, data is taken out of RAM and swapped to disk. If the process then tries to access that memory again, a <strong>major</strong> page fault occurs, the operating system has to retrieve the data from disk, load it into RAM, and then update the page table before the process can proceed.</p> <p>A minor page fault is when the data is still in RAM, but the page table for the process has not been mapped to the physical page yet. By default the operating system is lazy about mapping page table entries, but there are strategies for ensuring that all pages are mapped.</p> <h2 id="the-bottleneck"><a class="zola-anchor" href="#the-bottleneck" aria-label="Anchor link for: the-bottleneck"><i class="icon"></i></a> The Bottleneck</h2> <p>Every time you’re reading variables, or loading instructions, or saving data, the system is translating virtual memory addresses to physical addresses.</p> <h3 id="a-hardware-page-table-walk"><a class="zola-anchor" href="#a-hardware-page-table-walk" aria-label="Anchor link for: a-hardware-page-table-walk"><i class="icon"></i></a> A hardware page table walk</h3> <p>This translation can be quite involved if we dig really deep into the detail. The MMU looks at a register to find the physical address of the root page table. Then it takes some bits from the virtual address, goes to RAM, and reads the page table entry at that index. Then it takes the result from that lookup, goes back to RAM again, and finds the entry point to another intermediate table, and another intermediate table… until it gets the physical page frame number. It then combines the address with some of the bits from the original virtual address to find the exact byte in memory.</p> <h3 id="introducing-the-tlb"><a class="zola-anchor" href="#introducing-the-tlb" aria-label="Anchor link for: introducing-the-tlb"><i class="icon"></i></a> Introducing the TLB</h3> <p>The translation look-aside buffer (TLB) is a specialized cache within the MMU which remembers recent mappings to avoid the CPU having to check the full page table (in RAM) on every single memory access (billions of times per second)</p> <p>The consequence of a TLB miss is that the operation takes 100-200x longer. In reality this equates to about 100 nanoseconds, which is miniscule, but the volume of these operations causes that time to add up rapidly. Not to mention that 1 nanosecond would be much preferred.</p> <h2 id="how-to-make-money"><a class="zola-anchor" href="#how-to-make-money" aria-label="Anchor link for: how-to-make-money"><i class="icon"></i></a> How to make 💵 MONEY 💵</h2> <p>Since the consequence of TLB cache misses are so dire, it makes sense to avoid them. Working around the cache is out of the question, so the way forward is to enhance the effectiveness of the cache by making it have to cache less things, thereby improving its hit rate.</p> <p>The TLB has a limit on the number of page table entries it can hold. A page is typically 4KB but this may differ based on your hardware. So if your system has 32GB of memory, you need 8 million entries to map all the available memory on the system, which is not possible.</p> <h3 id="hugepages"><a class="zola-anchor" href="#hugepages" aria-label="Anchor link for: hugepages"><i class="icon"></i></a> Hugepages</h3> <p>This is where “Hugepages” come in.</p> <p>Hugepages is a feature in the linux kernel that allows a program to use much larger page sizes, usually either 2MB or 1GB. By using this option, you can effectively map all of the memory that you need such that it can fit in the limited number of entries in the TLB easily.</p> <p>For example, with the 32GB example from before, using 1GB hugepages, you would only need 32 entries, rather than 8,000,000.</p> <h4 id="enabling-hugepages-for-your-application"><a class="zola-anchor" href="#enabling-hugepages-for-your-application" aria-label="Anchor link for: enabling-hugepages-for-your-application"><i class="icon"></i></a> Enabling hugepages for your application</h4> <p>You could set a sysctl (<code>vm.nr_hugepages</code>), or you could control it from your program, for example…</p> <p>In C++, when calling <code>mmap</code>, you can pass the <code>MAP_HUGETLB</code> bitflag or <code>MAP_HUGE_1GB</code> for example.</p> <p>In Rust, there are some crates like <code>memmap2</code> or <code>hugepage-rs</code>.</p> <h3 id="memory-pinning"><a class="zola-anchor" href="#memory-pinning" aria-label="Anchor link for: memory-pinning"><i class="icon"></i></a> Memory Pinning</h3> <p>I mentioned earlier that when a process starts, the OS allocates virtual memory, but it lazily maps the page table entries. So you may have hugepages enabled, but you can still encounter a page fault if you access memory that hasn’t been mapped yet.</p> <p>There is a linux system call, <code>mlockall</code> which can be used to eagerly map all the virtual memory for your process. It’s advised to call this before entering the critical part of the program, such as any part of the program which needs to be real-time (requires deterministic timing).</p> <p>This system call not only “pre-faults” the pages so that they’re ready, it also pins them so that the operating system cannot swap them back to disk.</p> <h4 id="how-to-pin-memory-from-within-your-program"><a class="zola-anchor" href="#how-to-pin-memory-from-within-your-program" aria-label="Anchor link for: how-to-pin-memory-from-within-your-program"><i class="icon"></i></a> How to pin memory, from within your program</h4> <p>In C++ you would include the <code>sys/mman.h</code> header, allowing you to call mlockall directly.</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="c"><span class="giallo-l"><span style="color: #89DDFF;">#</span><span style="color: #BB9AF7;">include</span><span style="color: #89DDFF;"> &lt;</span><span style="color: #9ECE6A;">sys/mman.h</span><span style="color: #89DDFF;">&gt;</span></span> <span class="giallo-l"><span style="color: #89DDFF;">#</span><span style="color: #BB9AF7;">include</span><span style="color: #89DDFF;"> &lt;</span><span style="color: #9ECE6A;">iostream</span><span style="color: #89DDFF;">&gt;</span></span> <span class="giallo-l"><span style="color: #89DDFF;">#</span><span style="color: #BB9AF7;">include</span><span style="color: #89DDFF;"> &lt;</span><span style="color: #9ECE6A;">cstring</span><span style="color: #89DDFF;">&gt;</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #BB9AF7;">int</span><span style="color: #7AA2F7;"> main</span><span style="color: #9ABDF5;">() {</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> if</span><span style="color: #9ABDF5;"> (</span><span style="color: #7AA2F7;">mlockall</span><span style="color: #9ABDF5;">(</span><span style="color: #7AA2F7;">MCL_CURRENT </span><span style="color: #89DDFF;">|</span><span style="color: #7AA2F7;"> MCL_FUTURE</span><span style="color: #9ABDF5;">)</span><span style="color: #BB9AF7;"> !=</span><span style="color: #FF9E64;"> 0</span><span style="color: #9ABDF5;">) {</span></span> <span class="giallo-l"><span style="color: #9ABDF5;"> std::cerr </span><span style="color: #89DDFF;">&lt;&lt; &quot;</span><span style="color: #9ECE6A;">Failed to lock memory: </span><span style="color: #89DDFF;">&quot; &lt;&lt;</span><span style="color: #7AA2F7;"> std::strerror</span><span style="color: #9ABDF5;">(</span><span style="color: #7AA2F7;">errno</span><span style="color: #9ABDF5;">)</span><span style="color: #89DDFF;"> &lt;&lt;</span><span style="color: #9ABDF5;"> std::endl</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> return</span><span style="color: #FF9E64;"> 1</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #9ABDF5;"> }</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> munlockall</span><span style="color: #9ABDF5;">()</span><span style="color: #89DDFF;">;</span><span style="color: #9ABDF5;"> </span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> return</span><span style="color: #FF9E64;"> 0</span><span style="color: #89DDFF;">;</span></span> <span class="giallo-l"><span style="color: #9ABDF5;">}</span></span> <span class="giallo-l"></span></code></pre> <p>In Rust, you would use the <code>libc</code> crate and call it similarly.</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">use</span><span style="color: #0DB9D7;"> libc</span><span style="color: #89DDFF;">::{</span><span>mlockall</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> MCL_CURRENT</span><span style="color: #89DDFF;">,</span><span style="color: #C0CAF5;"> MCL_FUTURE</span><span style="color: #89DDFF;">};</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #89DDFF;">fn</span><span style="color: #7AA2F7;"> main</span><span style="color: #89DDFF;">() {</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> locked</span><span style="color: #89DDFF;"> = unsafe {</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> mlockall</span><span style="color: #89DDFF;">(</span><span style="color: #FF9E64;">MCL_CURRENT</span><span style="color: #BB9AF7;"> |</span><span style="color: #FF9E64;"> MCL_FUTURE</span><span style="color: #89DDFF;">)</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> };</span></span> <span class="giallo-l"></span> <span class="giallo-l"><span style="color: #BB9AF7;"> if</span><span style="color: #C0CAF5;"> locked</span><span style="color: #BB9AF7;"> !=</span><span style="color: #FF9E64;"> 0</span><span style="color: #89DDFF;"> {</span></span> <span class="giallo-l"><span style="color: #BB9AF7;"> let</span><span style="color: #C0CAF5;"> err</span><span style="color: #89DDFF;"> =</span><span style="color: #0DB9D7;"> std</span><span style="color: #89DDFF;">::</span><span style="color: #0DB9D7;">io</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">Error</span><span style="color: #89DDFF;">::</span><span style="color: #7AA2F7;">last_os_error</span><span style="color: #89DDFF;">();</span></span> <span class="giallo-l"><span style="color: #7AA2F7;"> panic!</span><span style="color: #89DDFF;">(&quot;</span><span style="color: #9ECE6A;">Failed to lock memory: </span><span style="color: #89DDFF;">{}&quot;,</span><span style="color: #C0CAF5;"> err</span><span style="color: #89DDFF;">);</span></span> <span class="giallo-l"><span style="color: #89DDFF;"> }</span></span> <span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre><h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="icon"></i></a> Conclusion</h2> <p>Ultimately, the illusion of memory that modern systems provide is a double-edged sword. While virtual memory addressing solves major problems around process isolation for most systems, the introduction of page tables and the need to perform multi-level page table walks introduces latency that is unnoticable to humans but harmful to low-latency and real-time applications.</p> <p>With this small bit of operating systems and linux knowledge you can use hugepages and memory pinning to reduce memory latency by up to 100 to 200 times in certain scenarios.</p> 2026-04-20T00:00:00+00:00 2026-04-20T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/linux-is-not-a-realtime-system/ <p><strong>Linux is not a realtime operating system.</strong></p> <p>This statement confused me at first, but as I looked further, it began to make sense.</p> <p>Linux provides an environment where multitasking is possible, facilitated by a scheduler which has goals such as minimizing wait times and latency, maximizing throughput, and maximizing fairness.</p> <p>These goals can sometimes pull in different directions. If you maximize throughput for one task, you increase latency for others. If you minimize latency by switching tasks more often, overheads start to pile up. If every process gets an equal slice of time for fairness, heavy tasks are starved while light tasks sit idle.</p> <p>Taking the general approach is the best decision for most usecases like servers and desktop computing, but it means that Linux is a “soft realtime” system.</p> <h2 id="defining-realtime"><a class="zola-anchor" href="#defining-realtime" aria-label="Anchor link for: defining-realtime"><i class="icon"></i></a> Defining “realtime”</h2> <p>What is evoked when you hear someone talk about something happening in <strong>realtime?</strong> We often think that it is happening immediately, like we’re describing something in the present.</p> <p>In computing, realtime refers to something more specific. From Wikipedia:</p> <blockquote> <p>A system is said to be realtime if the total correctness of an operation depends not only upon its logical correctness, but also upon the time in which it is performed.</p> </blockquote> <p>That is to say that it does not mean “fast” necessarily, it’s more to do with deterministic behavior in relation to time.</p> <p>Linux is a <strong>soft</strong> realtime system, meaning that if an operation misses its deadline, it isn’t a hard failure. Your process might wait a few hundred microseconds before the scheduler gives it a time-slice, and for the vast majority of cases, this is acceptable and nobody notices. If a deadline is missed frequently, the user experiences a degradation of service. For example, a web page might load slower, but still eventually load.</p> <p>A <strong>hard</strong> realtime system would be something like an assembly line, where if a part isn’t processed before its deadline, that part is no longer reachable by the robot, and the next robot won’t be able to process it correctly.</p> <h2 id="the-scheduler"><a class="zola-anchor" href="#the-scheduler" aria-label="Anchor link for: the-scheduler"><i class="icon"></i></a> The Scheduler</h2> <p>I mentioned earlier that Linux provides multitasking. It really provides the illusion of multitasking, at least on a single core. The task scheduler is what creates this illusion, by allocating slices of time to each task, interleaving their execution with one another to create the illusion of simultaneous progression. This is referred to as concurrency, as opposed to parallelism.</p> <small> Concurrency comes from the Latin "con" (together) and "currere" (to run, literally, to move faster than walking). </small> <p>Concurrency is great, especially when tasks have to wait for something to become available - the scheduler can pause that task until it’s ready to be executed, and give time-slices to other tasks in the meanwhile.</p> <p>The task itself can communicate to the scheduler that it should be paused to allow other tasks to progress. Often, a task will do this implicitly by executing a particular system call, such as <code>read()</code> or <code>sleep()</code>.</p> <p>However, the <em>scheduler</em> can also pause a task <em>without any cooperation from the task.</em></p> <img class="no-hover"alt="Scheduler standing behind the application, about to ruin its day"src="&#x2F;img&#x2F;preemption_meme_1.png"/><h2 id="preemption"><a class="zola-anchor" href="#preemption" aria-label="Anchor link for: preemption"><i class="icon"></i></a> Preemption</h2> <p>This particular style of multitasking on Linux is called preemptive multitasking. Tasks can be “preempted”.</p> <p>Sounds interesting, but what does it mean? <em>What</em> is being preempted?</p> <p>The current task’s execution time is being preempted - cut short - so that a more important task can be run.</p> <p>During preemption, the process state is saved, and another’s is loaded. This mechanism is what we know to be a context switch.</p> <p>An example where this is good: you run an application, and that application chews up all your compute, and you want to cancel it. If it couldn’t be preempted, you wouldn’t be able to move your cursor to close or cancel it.</p> <p>Where preemption becomes undesirable is when you’re in a low-latency environment such as high-frequency trading, and your trading application gets preempted and has to wait <strong>hundreds of microseconds</strong> to execute.</p> <h2 id="switch-it-off"><a class="zola-anchor" href="#switch-it-off" aria-label="Anchor link for: switch-it-off"><i class="icon"></i></a> Switch it off!!</h2> <p>You can use a tool like <code>chrt</code> to change how your running process is treated. For example, maybe you set its priority very high, in FIFO mode where it runs until it voluntarily yields, like so:</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="plain"><span class="giallo-l"><span>chrt --pid --fifo 99 &lt;PID&gt;</span></span></code></pre> <p><small>You can also check the current scheduling attributes with <code>chrt -p &lt;PID&gt;</code></small></p> <p>This will reduce the worst-case kernel latency but <strong>doesn’t get rid of all sources of jitter.</strong></p> <p>There are still things that can preempt the task. Here’s a few:</p> <h3 id="hard-interrupts-irqs"><a class="zola-anchor" href="#hard-interrupts-irqs" aria-label="Anchor link for: hard-interrupts-irqs"><i class="icon"></i></a> Hard interrupts (IRQs)</h3> <p>These are signals from hardware - network cards, disk controllers, or system timers. If your application runs on the same core that is handling interrupts, the CPU will pause your app to run the handler.</p> <p>You can examine the interrupts on an existing machine by inspecting <code>/proc/interrupts</code>. This will show you devices and their IRQ number.</p> <p>By default, all cores can service interrupts.</p> <blockquote class="tip"> <p class="alert-title"> <i class="icon"></i>Tip</p> <p>Non-essential hardware interrupts can be moved to another core or cores with interrupt affinity (<code>/proc/irq/&lt;IRQ_NUMBER&gt;/smp_affinity</code>). See <a rel="external" href="https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/html/performance_tuning_guide/s-cpu-irq">Interrupts and IRQ Tuning</a> in the Red Hat tuning guide</p> </blockquote> <h3 id="non-maskable-interrupts-nmi"><a class="zola-anchor" href="#non-maskable-interrupts-nmi" aria-label="Anchor link for: non-maskable-interrupts-nmi"><i class="icon"></i></a> Non-maskable interrupts (NMI)</h3> <p>An NMI is another hardware interrupt which the CPU cannot ignore or disable. These come from critical events like fatal hardware problems, memory errors, power failures, and watchdog timer timeouts.</p> <p>The watchdog is a daemon that keeps track of /dev/watchdog, which contains a heartbeat from a hardware timer on the motherboard. This is to detect CPU lockups.</p> <p>This can be and usually is disabled in HFT environments as far as I’m aware. This supposedly the most frequent cause of this type of interrupt. The other NMIs cannot be disabled and usually result in a kernel panic or system restart.</p> <blockquote class="tip"> <p class="alert-title"> <i class="icon"></i>Tip</p> <p>You can disable the watchdog while the system is running by setting the sysctl <code>kernel.nmi_watchdog</code> to <code>0</code>, or you can edit your bootloader such as GRUB with <code>nmi_watchdog=0</code> at the end of the line that starts with <code>GRUB_CMDLINE_LINUX_DEFAULT</code></p> <p>Occasionally it can show up in BIOS as “IPMI Watchdog Timer” which can be disabled.</p> </blockquote> <h3 id="system-management-interrupts-smi"><a class="zola-anchor" href="#system-management-interrupts-smi" aria-label="Anchor link for: system-management-interrupts-smi"><i class="icon"></i></a> System Management Interrupts (SMI)</h3> <p>This is also technically a non-maskable hardware interrupt, but it originates from the BIOS or EFI and pauses the entire OS. They’re particularly insidious because they cannot be detected via any standard linux observability.</p> <p>These interrupts occur when the motherboard needs to perform some kind of thermal throttling or power management, and can take in excess of 100 microseconds to complete.</p> <blockquote class="tip"> <p class="alert-title"> <i class="icon"></i>Tip</p> <p>There is nothing you can configure in the OS to prevent this, but there may be BIOS options like disabling “Global SMI” or “C-States”, and these options may only be present in server hardware that has been specifically optimized for HFT by the vendor.</p> </blockquote> <h3 id="software-faults"><a class="zola-anchor" href="#software-faults" aria-label="Anchor link for: software-faults"><i class="icon"></i></a> Software faults</h3> <p>If the trading app tries to access memory and encounters a page fault or a TLB miss, the kernel needs to take over and fetch the page.</p> <blockquote class="tip"> <p class="alert-title"> <i class="icon"></i>Tip</p> <p>Use huge-pages and pre-fault all pages. I’ll cover this more in a future article about the TLB specifically.</p> </blockquote> <h2 id="validation"><a class="zola-anchor" href="#validation" aria-label="Anchor link for: validation"><i class="icon"></i></a> Validation</h2> <p>There are some tools out there which can be used to run tasks and measure interference, such as:</p> <ul> <li><code>cyclictest</code> can be used to measure kernel scheduling latency</li> <li><code>hwlatdetect</code> can be used to measure latency caused by NMIs</li> </ul> <h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="icon"></i></a> Conclusion</h2> <p>The systems we use daily work seamlessly for us, but under the surface they’re executing sophisticated multitasking routines to give every process a fair chance at execution, with low-level hardware events stepping in and taking priority to avoid abrupt system failures.</p> <p>In a trading environment, without knowledge of these interactions, your application would be subject to all kinds of jitter, causing you to miss a “fill” or to get sniped on a fast-moving order book…</p> <p>… and that can be the difference between profit and poverty.</p> 2026-04-17T00:00:00+00:00 2026-04-17T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/why-numa-is-important-in-trading/ <p>While modern software tries to hide hardware complexity behind abstractions, staff working in HFT must dig down to ensure that every nanosecond is spent executing trading logic, not chewed up by operating system overheads or noisy neighbouring processes.</p> <p>Learning the ins and outs of NUMA is not a micro-optimization, it is a requirement.</p> <h3 id="short-history-of-numa"><a class="zola-anchor" href="#short-history-of-numa" aria-label="Anchor link for: short-history-of-numa"><i class="icon"></i></a> Short history of NUMA</h3> <p>Back in the day, CPUs were slower than memory. In the 1960s, processors began to overtake memory in speed and as a result found themselves stuck waiting for data to arrive from a memory access.</p> <p>This was solved by either avoiding memory access, or by adding cache memory some of which you may have heard of like L1 and L2 cache.</p> <p>However, as operating systems and applications have grown, these caches are no longer as effective. Additionally, on modern servers with multiple CPU sockets, only one processor can access the memory at one time, causing the other to wait.</p> <p>Non-uniform memory access (NUMA) was created to solve this problem by providing separate memory to each socket.</p> <h3 id="why-numa-must-be-considered-in-a-high-frequency-trading-hft-environment"><a class="zola-anchor" href="#why-numa-must-be-considered-in-a-high-frequency-trading-hft-environment" aria-label="Anchor link for: why-numa-must-be-considered-in-a-high-frequency-trading-hft-environment"><i class="icon"></i></a> Why NUMA must be considered in a high-frequency trading (HFT) environment</h3> <p>In HFT, nanoseconds matter. If your application has to wait an extra amount of nanoseconds every time it needs to access some memory, it quickly adds up into microseconds and perhaps even milliseconds.</p> <p>On hardware with NUMA, data could reside in non-local memory aka the memory of the other socket on the server board. For the local core to access this memory it has to traverse an interconnect which can take 100-200 nanoseconds.</p> <h3 id="how-to-protect-your-application-from-the-downsides-of-numa"><a class="zola-anchor" href="#how-to-protect-your-application-from-the-downsides-of-numa" aria-label="Anchor link for: how-to-protect-your-application-from-the-downsides-of-numa"><i class="icon"></i></a> How to protect your application from the downsides of NUMA</h3> <p>Firstly, on Linux, NUMA is modelled as nodes. For example, Node 0 is Socket 0 plus its attached memory, for however many sockets your system has.</p> <p>You can configure different policies per-thread or per-allocation.</p> <ul> <li><code>local</code> (default) - allocate on the node where the thread is running.</li> <li><code>bind</code> - allocate <em>only</em> on specified nodes.</li> <li><code>preferred</code> - try a particular node first but fallback to others.</li> <li><code>interleave</code> - round-robin across nodes.</li> </ul> <p>In HFT you probably want to use <code>bind</code>.</p> <h4 id="discover-your-topology"><a class="zola-anchor" href="#discover-your-topology" aria-label="Anchor link for: discover-your-topology"><i class="icon"></i></a> Discover your topology</h4> <p>Even on your laptop or desktop linux computer, you should be able to install <code>numactl</code>, <code>lscpu</code>, or <code>lstopo</code> and look at the nodes.</p> <p>This is my desktop for example, which has one socket with eight cores, and 32 GB of memory:</p> <img class="no-hover"alt="CPU topography"src="&#x2F;img&#x2F;lstopo.png"/> <p>On a server system with multiple cores you should see additional separate NUMA nodes.</p> <h4 id="disable-automatic-numa-balancing"><a class="zola-anchor" href="#disable-automatic-numa-balancing" aria-label="Anchor link for: disable-automatic-numa-balancing"><i class="icon"></i></a> Disable automatic NUMA balancing</h4> <p>Automatic balancing is done by <code>autonuma</code> which scans processes’ memory and migrates pages to try to optimise locality, which is fine for most cases but can cause latency spikes that are unacceptable in a trading environment.</p> <p>There are two things that must be done to disable balancing.</p> <p>Adjust the following sysctl:</p> <pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="shellscript"><span class="giallo-l"><span style="color: #0DB9D7;">echo</span><span style="color: #FF9E64;"> 0</span><span style="color: #89DDFF;"> &gt;</span><span style="color: #9ECE6A;"> /proc/sys/kernel/numa_balancing</span></span></code></pre> <p>and then add <code>numa=off</code> to the boot-loader configuration, or similar, depending on your boot-loader.</p> <h4 id="pin-your-application"><a class="zola-anchor" href="#pin-your-application" aria-label="Anchor link for: pin-your-application"><i class="icon"></i></a> Pin your application</h4> <p>Use <code>numactl --cpunodebind=&lt;node&gt; --membind=&lt;node&gt; &lt;the_program&gt;</code></p> <p>It should be noted that the scheduler can still move threads between cores on that node, so in addition to this you’ll want to pin your application to the same cores using <code>taskset</code> or similar, like <code>taskset -acp &lt;cores&gt; &lt;pid&gt;</code></p> <h4 id="validate-the-setup"><a class="zola-anchor" href="#validate-the-setup" aria-label="Anchor link for: validate-the-setup"><i class="icon"></i></a> Validate the setup</h4> <p>Check that your process is actually bound correctly with <code>numastat -p &lt;pid&gt;</code>. This will print out a table detailing its memory usage for each node, where the node should only be the one you’ve bound the process to.</p> <h3 id="what-if-my-systems-engineers-misconfigure-numa"><a class="zola-anchor" href="#what-if-my-systems-engineers-misconfigure-numa" aria-label="Anchor link for: what-if-my-systems-engineers-misconfigure-numa"><i class="icon"></i></a> What if my systems engineers misconfigure NUMA?</h3> <p>If you’re a developer in a trading firm and you’ve deployed your application, you can still check NUMA at runtime and ensure your allocations land on the correct node. The library <code>libnuma</code> has an API that allows you to check if NUMA is available, how many nodes there are, which node a given core belongs to, and the ability to allocate on a particular node.</p> <p>In reality, any competent linux/production/systems/sre team in a HFT should already be working with their developers to ensure everything it set up correctly.</p> <h3 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="icon"></i></a> Conclusion</h3> <p>In the overwhelming majority of environments NUMA goes by unnoticed, with the defaults being suitable for most workloads. However in trading, if you ignore NUMA, the results can be catastrophic. Microseconds of latency across millions of memory accesses can be the difference between profit and poverty.</p> 2026-03-25T00:00:00+00:00 2026-03-25T00:00:00+00:00 Vasilios Syrakis https://cetanu.github.io/blog/a-philosophy-of-software-design/ <p>Today I finished reading the book, “A Philosophy of Software Design”, by John Outsterhout, a professor at Stanford University.</p> <p>I wanted to write a little about what I learnt from the book, plus how I see it mapping to my experience in my work where I’ve dealt with software and choices around design.</p> <h3 id="complexity-is-incremental"><a class="zola-anchor" href="#complexity-is-incremental" aria-label="Anchor link for: complexity-is-incremental"><i class="icon"></i></a> Complexity is incremental</h3> <p>This is probably obvious to anyone who has worked on software in any serious capacity, but (unless you’re vibe-coding) complexity doesn’t just show up in a single pull-request.</p> <p>It’s the “Boiling Frog” problem, except you’re not the frog, you’re the chef…<br /> If we increase the temperature by just one degree, surely it’s not that bad, right?<br /> This is how we justify adding complexity to ourselves when we say “it works for now, I’ll clean it up later” on a pull-request.</p> <p>The incremental quality of complexity is what makes it so insidious, because it’s so easy to trade a small bit of complexity to get something delivered. You can even continue to make that trade-off, again and again, for years. Who cares if an abstraction leaks a little bit, as long as I can fix up this edge case right now?</p> <p>It’s hard to pinpoint the exact moment when things become sluggish, hard to change, prone to errors, and difficult for new engineers to understand. That’s because there’s no exact moment.<br /> Just like how a frog doesn’t suddenly start boiling when the water goes from 99°C to 100°C.</p> <h4 id="it-s-shared-discipline-not-a-technical-challenge"><a class="zola-anchor" href="#it-s-shared-discipline-not-a-technical-challenge" aria-label="Anchor link for: it-s-shared-discipline-not-a-technical-challenge"><i class="icon"></i></a> It’s shared discipline, not a technical challenge</h4> <p>I might be totally wrong on this, but I get a general sense that when people think about complexity, the focus is on technical decisions that get made over time. It almost implies that you can solve complexity all by yourself if you just work hard enough.</p> <p>We work with other people. I believe it’s more pragmatic to shift this perception so that the focus is on person-to-person interactions - which I assert is where the problem arises.</p> <p>When a reviewer fails to stand their ground, complexity is welcomed in. This can be tricky, because often you’re standing against a peer who you trust, who means well, but who might be too busy to spend the extra time to come up with a design that doesn’t attach one more tiny anchor to the codebase.</p> <p>I comfort myself by thinking that nobody intentionally adds complexity. They can’t see a better way, or maybe they didn’t ask themselves if there could be a better design. Thinking takes energy and I don’t blame anyone for taking the path of least resistance to get their work done. Perhaps what we need is better ways to measure complexity, so that we can reward its reduction.</p> <p>For people working solo, you are both the author and reviewer. If you aren’t trying to see your own work from a different perspective, then your “reviewer” persona is effectively rubber-stamping the work, shrugging, and walking away.</p> <h3 id="abstractions-are-about-hiding-unnecessary-details"><a class="zola-anchor" href="#abstractions-are-about-hiding-unnecessary-details" aria-label="Anchor link for: abstractions-are-about-hiding-unnecessary-details"><i class="icon"></i></a> Abstractions are about hiding unnecessary details</h3> <p>This one probably sounds like common sense. Duh! That’s what an abstraction is.</p> <p>However, for me, I think it’s valuable to consider that you and I sometimes make abstractions that accidentally hide <em>necessary</em> details.</p> <p>This is one of those things where, if you zoom out a little bit, makes software engineering seem like such a challenging field. The subjectivity of this - where to draw the boundaries of an abstraction - highlights a constant cognitive requirement towards making a balanced decision.</p> <p>You can’t open a book and have it tell you how to make the right abstraction. You just have to fail, walk into a mess, learn from the mess, and make the next project better, only to fall into another mess. A mess that grows as it feeds on entropy and junior developers.</p> <h3 id="deep-modules-are-simpler-than-shallow-modules"><a class="zola-anchor" href="#deep-modules-are-simpler-than-shallow-modules" aria-label="Anchor link for: deep-modules-are-simpler-than-shallow-modules"><i class="icon"></i></a> Deep modules are simpler than shallow modules</h3> <p>First, what does this even mean? What is a deep module?<br /> A module is deep when its implementation is large compared to its interface.</p> <p>That means that you could have a module that is long (in terms of lines of code), but if every function in that module is exported, the interface is large.</p> <p>Instead, a deep module would be something with perhaps a few key functions or classes exported, which encapsulate the implementation behind a concise interface.</p> <p>This was something that surprised me at first but makes total sense. If make a library, I don’t want the caller to have to know about 20 different functions or methods that they need to call, especially if most of the time all of the arguments passed in are very similar.</p> <h3 id="general-purpose-code-is-simpler-than-special-cased-code"><a class="zola-anchor" href="#general-purpose-code-is-simpler-than-special-cased-code" aria-label="Anchor link for: general-purpose-code-is-simpler-than-special-cased-code"><i class="icon"></i></a> General-purpose code is simpler than special-cased code</h3> <p>What exactly is the difference between the two?<br /> General purpose code solve a class of problems. Special case code solves one problem.</p> <p>Example:</p> <ul> <li>Special case: A function that adds the <code>Authorization</code> request header with a particular value</li> <li>General purpose: A function that adds a request header, with a given key and value.</li> </ul> <p>Special cases pop up in codebases after their initial design has been implemented, a bunch of users have been onboarded, it’s operating fine, but one particular (and important) user needs it to do something that wasn’t in the original design. This is expected in software, and rigidity isn’t a good response.</p> <p>If you immediately know how to change the system so that their particular need is solved as a general case it’s a clear win, but this doesn’t seem to be how things go in most cases. Instead, it takes painful thinking to determine what needs to be rearranged, migrated, and mutated, in order for the special case to be handled in such a way that it doesn’t require someone to carve out time from their day to maintain it.</p> <h2 id="philosophy-not-science"><a class="zola-anchor" href="#philosophy-not-science" aria-label="Anchor link for: philosophy-not-science"><i class="icon"></i></a> Philosophy, not Science</h2> <p>There is a particular theme that seems to be common across the above concepts.</p> <ul> <li>What makes a good abstraction?</li> <li>How deep should a module be?</li> <li>How general should my code be?</li> </ul> <p>There’s no formula. A book can’t teach it to you. A committee can’t develop a standard for it.</p> <p>The fundamental question being wrestled with is “Where do I draw the boundary?” and the answer is different from wherever you are standing.</p> <p>It’s the human factors that help answer this question. How much cognitive load can the user or reader bear? How often will I have to change the codebase? A compiler or linter can’t answer these questions. It requires a person to draw a subjective line in the sand and then defend it.</p> <h3 id="the-element-of-taste"><a class="zola-anchor" href="#the-element-of-taste" aria-label="Anchor link for: the-element-of-taste"><i class="icon"></i></a> The element of “Taste”</h3> <p>I hear about how the future of software will be influenced by having good “taste” now that LLMs are writing all the code (I don’t actually believe this).<br /> If that’s true then I sincerely hope that all the people that intend to map their taste into software learn good software design first.</p> <p>Ironically, the way to learn good software design is through experience. You have to ship something and collect the feedback. Sometimes that feedback is customers telling you what causes friction. Other times it shows up in the volume of questions directed at a particular feature… at 3AM after your site blows up.</p> <p>While reading the book you get a sense that it’s never going to answer this question, “What’s a good abstraction?” because it can’t. The professor encourages people to find the right balance for themselves, and it introduces distinctions that might enable them to do so.</p> <p>You might be creating a good design if the abstraction aligns with a business domain, or when you provide a neat general-purpose interface across an entire system. You might be creating a bad design when you add logic that purely addresses an immediate need and nothing else.</p> <h2 id="my-closing-thought"><a class="zola-anchor" href="#my-closing-thought" aria-label="Anchor link for: my-closing-thought"><i class="icon"></i></a> My closing thought</h2> <p>Writing this page made me realise that these problems that I found unapproachable - how to decide where to draw the line - are simply the way things are. It’s the essence of design.</p> <p>Just like other art forms, you can practice, fail, learn, and improve, but there will always be some people that either love or hate the art that you created.</p> <p>Often that person is your own self. The inner-critic looks at the past and is ashamed of mistakes. I choose to forgive myself for the imperfections and focus on the parts that I’m proud of.</p>