Caching: Put Data Closer, Get Speed — and New Problems

What Problem Is Caching Actually Solving

Caching means copying data that's read often but rarely changes into a faster, cheaper storage layer, so future requests can hit that copy directly instead of going back to the origin — usually a database — to recompute or re-query every single time. The upside is pretty direct:

• Lower latency: reading from memory or something like Redis is orders of magnitude faster than hitting a database or running a complex computation.
• Less load on the origin: repeated queries get intercepted by the cache, so the actual traffic that reaches the database drops sharply.

Sounds simple enough. But the moment data lives in two places — the cache and the origin — a classic problem shows up: when does the cached copy drift out of sync with the origin, and what do you do about it when it does? That's where caching actually gets hard. It's not about how you store the data — it's about how you keep it fresh.

Where Can You Put a Cache

Between a user's request and the database, you can actually slot a cache into pretty much every layer:

• Client-side cache: the browser stores static assets (images, JS files) locally, controlling how long they stay valid through HTTP headers like Cache-Control and ETag.
• CDN (Content Delivery Network): copies static content to nodes physically closest to the user, so cross-region requests never have to travel back to the origin server.
• Application-layer cache: an in-memory store like Redis or Memcached sits between your app servers and the database, holding query results or computed data.
• Database-level caching: most databases have a built-in query cache of their own, speeding up repeated queries.

The closer a cache layer sits to the user, the more latency it saves — but the data it can hold also tends to be simpler and more static. The application layer is the one we usually end up designing ourselves, and it's what the rest of this post — the read and write strategies below — is really about.

Read Strategies: Cache-Aside and Read-Through

The most common read strategy is Cache-Aside, where the application owns the entire logic itself:

// Cache-Aside: check the cache first, fall back to the database on a miss, then backfill the cache
async function getProduct(productId, cache, db) {
  const cached = await cache.get(`product:${productId}`)
  if (cached) return cached // cache hit
 
  const product = await db.query('SELECT * FROM products WHERE id = ?', productId)
  await cache.set(`product:${productId}`, product, { ttl: 300 }) // backfill the cache, expire in 300 seconds
  return product
}

The first query is a cache miss, so it has to go hit the database for real — but once it gets a result back, it writes that result into the cache, so the same query hits the cache directly from then on. This pattern is simple and easy to reason about. The downside: whoever makes that first query eats the latency of populating the cache, and the application has to remember to write back every time. Forget that one line, and the cache just stays empty forever.

Read-Through hands that "go fetch it from the source on a miss" responsibility to the cache layer itself — the application only ever talks to the cache and doesn't need to know there's a database behind it. Conceptually it's close to Cache-Aside; the only difference is who's responsible for backfilling. Whether it's available in practice depends on the caching product you're using.

Write Strategies: Write-Through, Write-Behind, and Write-Around

Reads solve the "how do I get it" problem. But when data changes, how the cache keeps up is a whole separate set of tradeoffs.

Write-Through: every write goes to both the cache and the database at the same time, and it only counts as successful once both confirm. The upside is that the cache and database almost never drift apart. The cost: every write now pays the extra latency of a cache write too.

Write-Behind (also called Write-Back): a write lands in the cache and returns success immediately, and a background job batches those changes and flushes them to the database asynchronously. The upside is fast writes. The cost is a window where the cache and database disagree — and if the cache dies before that background flush happens, whatever changed in that window is just gone. It's the same tradeoff as asynchronous replication in the replication post: you're trading consistency for performance.

Write-Around: writes go straight to the database and never touch the cache — the cache only gets populated passively, Cache-Aside style, when something actually reads that data. This fits data that isn't read right after it's written, and it keeps the cache from filling up with stuff nobody ends up using.

Cache Invalidation: The Actually Hard Part

Cache invalidation is hard because a cached value is, by definition, a snapshot from some point in the past — and the origin could have already changed. There are two common ways to deal with it:

• TTL (Time To Live): every cached entry gets an expiration time; once it's up, the entry goes stale automatically and the next read pulls a fresh copy from the origin. Simple to implement, but picking the right TTL is its own art — too short and the cache barely does anything, too long and users might be staring at data that's badly out of date.
• Active invalidation (event-based invalidation): the moment data gets written or updated, you delete or update the matching cache entry right then, instead of waiting for it to expire naturally. Data reflects the latest state faster, but now the application has to remember to touch the cache at every place that touches the data — that logic gets scattered everywhere, and it's easy to miss a spot.

Most real systems use both: active invalidation handles the cases you know will change, and TTL acts as a safety net — even if you miss an active invalidation call somewhere, the data won't stay wrong forever.

Eviction Policy: Who Gets Kicked Out When the Cache Fills Up

Cache capacity is finite. Once it's full and new data needs to come in, something has to get evicted — that's what an eviction policy decides. A few common approaches:

• LRU (Least Recently Used): evicts whatever hasn't been read in the longest time, on the assumption that data nobody's touched recently probably won't be needed again soon. This is the default policy for most caching systems.
• LFU (Least Frequently Used): evicts whatever's been read the fewest times, regardless of how recently.
• FIFO (First In, First Out): evicts in the order entries were added — whatever went in first goes out first, whether or not it's ever been read.

Which policy makes sense depends on your access pattern. For something like product pages, where popular items stay popular, LRU usually does well. If access patterns are more random, the choice of eviction policy matters a lot less.

Cache Stampede: When One Expiration Slams the Database

Say the cache entry for some hot piece of data expires right as a thousand requests land in the same second. Under plain Cache-Aside logic, every single one of them sees a cache miss and rushes straight to the database at once — a spike completely out of proportion to the traffic the database normally sees once the cache is warm again, and it can knock the database over. This is called a cache stampede (also known as the thundering herd problem).

Common ways to soften this:

• Mutex lock: on a cache miss, only let the first request through to query the database and refill the cache; everyone else either waits or serves stale data in the meantime, and once that first request finishes, they all get a hit.
• Jitter (early, randomized expiration): add a small random offset to the TTL so a big batch of entries doesn't expire in the same instant — it spreads out when things go stale.
• Probabilistic early expiration: just before an entry is due to expire, a background job refreshes it probabilistically ahead of time, instead of waiting for the actual expiration moment to trigger a scramble.

The shared idea behind all of these: never let a cache expiring and a flood of requests land at the exact same moment.

Which One Should You Pick? It's Still About Tradeoffs

By now the pattern should be clear: every caching decision trades one cost for another. Cache-Aside is simple, but the application owns the backfill responsibility. Write-Through keeps things consistent, but writes get slower. TTL is easy to maintain, but staleness isn't guaranteed to line up with reality. Just like the tradeoffs in idempotency and replication, there's no universally-best caching strategy. Which read/write pattern to use, how long to set the TTL, whether to bother handling stampedes — all of that comes back to your actual read/write ratio and how much staleness you can tolerate, not whatever settings someone else used.

Wrapping Up

Caching copies frequently-read data into a faster storage layer, trading that for lower latency and less load on the origin — and in exchange, you take on the risk of the cache and the origin disagreeing. On the read side, Cache-Aside lets the application control exactly when the cache gets refilled. On the write side, you're choosing between the consistency of Write-Through and the performance of Write-Behind. The genuinely hard part is cache invalidation, where TTL and active invalidation have to back each other up. And figuring out what to evict when the cache fills up, or how to soften a flood of requests hitting an expired cache at once, is what turns caching from "just add a storage layer" into an engineering problem that deserves careful design.