Vasilios Syrakis - hardwareA simple blog made with Zola and DuckquillZola2026-07-07T00:00:00+00:00https://cetanu.github.io/tags/hardware/atom.xmlProfit or Poverty: False Sharing2026-07-07T00:00:00+00:002026-07-07T00:00:00+00:00
Vasilios Syrakis
https://cetanu.github.io/blog/false-sharing/<p>In previous blogs I talked a little bit about
<a href="https://cetanu.github.io/blog/why-numa-is-important-in-trading/">NUMA</a> and the
<a href="https://cetanu.github.io/blog/translation-lookaside-buffer/">TLB</a>, if you haven’t read those, I
recommend starting there before reading this.</p>
<p>You should already understand that accessing main memory is relatively slow for
the CPU, especially in a low-latency environment, and you may be familiar with
the fact that the CPU has small caches of varying sizes on the chip, known as
the L1, L2, and L3 caches.</p>
<p>As their name implies, there are levels, or a hierarchy, from fastest (and
smallest) to slowest (and largest). The L1 cache is also typically split into
two caches, one being an instruction cache and the other being a data cache.</p>
<p>Fun fact: these caches are usually made of SRAM and usually they are the
largest part on the chip.</p>
<img class="no-hover"alt="relative speed of hardware"src="/img/relative-hw-speed.png"/><h2 id="requesting-data-from-the-cache"><a class="zola-anchor" href="#requesting-data-from-the-cache" aria-label="Anchor link for: requesting-data-from-the-cache"><i class="icon"></i></a>
Requesting data from the cache</h2>
<p>You may have heard of “cache lines” before (also called cache blocks), maybe on
some article about performance on hackernews.</p>
<p>A cache line can be thought of a single unit of data which is transferred from
RAM to the cache. Usually one cache line is 64 bytes, but this can differ
depending on your CPU architecture.</p>
<p>This introduces the concept of cache locality. When the CPU needs to access the
same memory several times in a row, after it’s already been pulled into the L1
cache, this is referred to as <strong>temporal locality</strong>. When the CPU pulls in a
cache line and all the data it needs is within that block, it’s referred to as
spatial locality. A good example of <strong>spatial locality</strong> is when you have an
array that fits into one cache line, resulting in traversal being extremely
fast. An example of potentially terrible cache performance might be something
like a linked-list.</p>
<p>On systems with multiple CPU cores, they might share the same chunk of memory
in their local caches. Most processors use what is known as the MESI protocol
to prevent overwriting each others data. Jon Gjengset has a <a href="https://cetanu.github.io/blog/false-sharing/www.youtube.com/watch?v=tND-wBBZ8RY">great
talk</a> about Mutexes which also covers the
MESI protocol.</p>
<h2 id="where-it-can-go-wrong"><a class="zola-anchor" href="#where-it-can-go-wrong" aria-label="Anchor link for: where-it-can-go-wrong"><i class="icon"></i></a>
Where it can go wrong</h2>
<p>CPUs can enter what is called a coherence livelock or cache-line pingponging,
where multiple cores fight over the same memory, or the same cache line.</p>
<p>The first way this can happen is called <strong>False Sharing</strong>, where your data may
reside in a cache line, but it may not be occupying that entire cache line by
itself. If your data sits in the first half but some other data sits in the
latter half, it’s possible that two CPUs may continuously write to the same
cache line, requiring them to communicate with each other to gain ownership in
order to modify the cache line. If you watched Jon’s video above you would know
that this can eat up dozens of precious nanoseconds, just for the comms between
CPUs. As you increase the number of cores on a system this problem can be
exacerbated.</p>
<img class="no-hover"alt="diagram of false sharing"src="/img/false-sharing.png"/>
<p>The second way is <strong>True Sharing</strong>, where cores may write to the same variable or
data structure in memory. This could happen during atomic operations or when
using synchronization primitives such as mutexes, semaphores, barriers,
spinlocks, and so on.</p>
<h2 id="how-to-protect-against-it"><a class="zola-anchor" href="#how-to-protect-against-it" aria-label="Anchor link for: how-to-protect-against-it"><i class="icon"></i></a>
How to protect against it</h2>
<p>At least for false sharing, the solution seems simple: just pad your data
structure or variable so that it occupies an entire cache line. Obviously it
would be silly to do this with every variable in your program, so you have to
pick the time-critical parts of your application here.</p>
<pre class="giallo" style="color: #A9B1D6; background-color: #1A1B26;"><code data-lang="rust"><span class="giallo-l"><span style="color: #89DDFF;">use</span><span style="color: #0DB9D7;"> std</span><span style="color: #89DDFF;">::</span><span style="color: #0DB9D7;">sync</span><span style="color: #89DDFF;">::</span><span style="color: #0DB9D7;">atomic</span><span style="color: #89DDFF;">::</span><span style="color: #C0CAF5;">AtomicU64</span><span style="color: #89DDFF;">;</span></span>
<span class="giallo-l"></span>
<span class="giallo-l"><span style="color: #89DDFF;">#[</span><span>repr</span><span style="color: #89DDFF;">(</span><span>align</span><span style="color: #89DDFF;">(</span><span>64</span><span style="color: #89DDFF;">))]</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">pub</span><span style="color: #BB9AF7;"> struct</span><span style="color: #C0CAF5;"> PaddedAtomic</span><span style="color: #89DDFF;"> {</span></span>
<span class="giallo-l"><span style="color: #89DDFF;"> pub</span><span style="color: #C0CAF5;"> value</span><span style="color: #89DDFF;">:</span><span style="color: #C0CAF5;"> AtomicU64</span><span style="color: #89DDFF;">,</span></span>
<span class="giallo-l"><span style="color: #89DDFF;">}</span></span></code></pre>
<p>For true sharing, it seems quite a bit more complicated to get right. Sometimes
you really do need atomic operations on global memory.</p>
<p>You could pull data into thread local storage, only merging back at the end of
expensive or time critical execution.</p>
<p>You could reorganize your program to do batching to reduce how often it needs
to write back to the shared memory.</p>
<p>Maybe a lock-free data structure could be used.</p>
<p>The solution most likely depends on exactly what you are willing to accept in
terms of trade-offs.</p>
<h2 id="conclusion"><a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion"><i class="icon"></i></a>
Conclusion</h2>
<p>Like I usually say, for the most part things like this are unnoticeable and
work just fine for the majority of situations. If you’re running a flask webapp
you probably don’t and should not care about this.</p>
<p>When you require low-latency, then you need to understand how to write your
programs such that they exploit cache-lines for your architecture because it
makes the difference between a 1ns L1 cache hit, or falling off a cliff with a
100x slower execution.</p>