Cache-Aside in Production: One Read Pattern, Two Invalidation Strategies

ache invalidation strategies

Today we’re going a bit deeper into Cache-Aside, this pattern that’s pretty well known and popular across blogs and websites. The idea is simple: the client hits the app, the app checks if the data it’s asking for is already in cache. If it is, it returns it. If not, it goes to the database, returns the data, and stores it in cache so the next user doesn’t have to wait as long for a response. Simple enough so far.

But here’s the thing: a pattern can be shared and still need opposite invalidation strategies. Here are two cases:

Case 1: an external API that never tells you when its data changes

Say we have an analytics API and we need data from the last 30 days. The goal is to load it fast and avoid asking the API again if we already have the data. With cache-aside, we check the cache, and on a miss, we call the API, store the response with a TTL, and return it.

This is the right approach for this scenario, but it comes with costs we’re accepting on purpose:

  • More keys in memory, bounded by the TTL.
  • Analytics APIs often revise the most recent days after first reporting them, so a long TTL costs more at the end of the range than at the start. Check your provider’s documentation before choosing a TTL.

When not to use it?

When freshness matters more than quota, when queries are varied enough that the hit rate stays low, or when the provider gives you a way to get notified so you know when to invalidate.

Case 2: related posts, invalidated lazily

What we implemented: say we have a WordPress site with a related posts section, calculated from shared tags. To handle it, we store it in an object cache backed by Redis.

Without this, posts getting heavy traffic would be hitting the database with the same repeated queries. Also worth noting: WordPress’s object cache is not persistent by default, which is why we’re relying on Redis here.

Each entry only stores post IDs. Titles, images, and excerpts are pulled fresh at render time, and the render only returns published posts.

Here the source is your own database, so you do get an event when a post gets saved (through an action hook, for example). The hard part is finding the list (or lists) that contain the post you just edited, and doing that inside the save request would turn into a performance problem down the road. With 10 or 15 posts, no big deal, but imagine something like the New York Times, saving a post would be a nightmare.

To get around this, we can push that work to the read side instead:

  1. Add a save hook that adds the edited post’s ID to a temporary set in the object cache. This set sticks around for at least a week, or as long as that post’s own cache entry exists.
  2. Then, when the related posts list gets read, you check it against that saved set.
  3. If there’s a match, the list gets rebuilt from the database and you remove the ID from the set.

But this approach isn’t 100% effective for every scenario. Here are the costs and failure points it brings:

  • The first reader after an edit pays for the rebuild.
  • New posts show up eventually, when a list expires or is evicted along with its post’s cache entry and gets recalculated. There is no upper bound on that delay: by default, entries are stored with no expiration, so they live until they are evicted.
  • Removing the ID after the first rebuild leaves other lists that held the same post with their old membership until they are rebuilt. Filtering unpublished posts at render time hides the worst case, but those lists can be shorter than they should be.

So, where should you use it?

On a site with way more reads than writes, and tolerance for eventual consistency.

When not to use it?

When a new post needs to show up immediately, or when an incomplete or outdated list isn’t acceptable.

So what fits your scale?

The approach above doesn’t solve the problem at scale, it’s suited to a specific range of traffic, more than a specific site size. To figure out when it stops working, there are 2 variables to look at: how often your busiest tag gets edited, and how many lists a single post can land in (its fan-out).

Why it holds up for low to medium traffic sites

Since edit volume is low, the set of pending IDs stays small, and the leak we covered above rarely compounds into anything visible.

When does it break at scale? (tens of thousands of posts, constant publishing, posts referenced by thousands of lists)

Once you’re dealing with a site at this level of traffic and activity (think a New York Times), the invalidation trick stops working, so related posts need to be pulled out into their own service (search or recommendation infrastructure, recalculated asynchronously and in batches). The app then reads from that service with cache-aside and a short TTL, basically the same pattern as Case A.

Comments

Leave a Reply