When to use a Cache

Discussion about when a Cache system should be added to an application. It includes a calculator to help developers arriving this site.

Short answer

Should I use a Cache to solve my current problem?

No, but this answer might be wrong because it was cached ;)

Cache Optimization Calculator

Long answer

When anyone says about adding a cache to make any service faster I can only ask: “how faster is it going to be?”. Usually I have no answer, just people angry with me.

But that’s just because they didn’t read this article or just didn’t stop to think about that.

Let’s say you want to cache a search in a database. So the first thing you should do is to measure how long does it take to perform that query, as average.

Perhaps you will realize it is fast enough and it doesn’t worth it to create the cache.

If even then you think that the cache does worth it, then it is important to measure:

• how long does the Cache system take to read data.
• how long does the Cache system take to write data.
• success ratio for the cache, or how many times it will find data in cache.

It is important to add those measures to our monitoring system.

Then we can apply this formula: The time spent by our initial system was just the time to read a value from database, but with a cache it will be:

$$ Cache_{read} + ((1 - S_R) * (Op + Cache_{write})) $$

Where:

• \(S_R\) is the Success Ratio. It is a proportion between 0 and 1.
• \(Cache_{read}\) is the Cache read time.
• \(Cache_{write}\) is the Cache write time.
• \(Op\) is the Operation time to be improved. It might be a database access, a function call, …

As code

Perhaps reading the usual algorithm it is easier to understand the previous formula:

1
2
3
4

value = retrieve_from_cache(key)        # Cache Read
if value is None:                       # Will enter (1 - SuccessRate) times
    value = operation()                 # Main operation to be performed
    store_in_cache(key)                 # Cache Write

The pricing using Cache

An additional service

Having a cache usually requires to use an additional service, such as Redis, Memcached, …

This service requires to be configured with High Availability (HA), with several hosts, what costs money and time. It requires maintenance, updates, …

A reader can say: “That’s not true, because I will reuse the installation I already have”. Might be, but you will require to keep in mind the new CPU, RAM and Disk requirements for the new service and how it could affect the existing ones.

Cache invalidation

When we use a Cache system for data that might change with time we will add a new problem: Cache invalidation. It is really hard or even impossible to deal with it with no errors.

It is not enough to send an invalidation messagge. If our Cache system is composed by several instances, we need to ensure that:

• all of them have been notified
• no other request for the same value is running in parallel that could retrieve the old value and store it in Cache.

According to Martin Fowler:

There are only two hard things in Computer Science: cache invalidation and naming things

Monitoring

When using a cache the monitoring requests will increment because we need to measure how well is the cache doing.

So, previous code should turn into:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

with measure_time("cache_read"):
    value = run_retrieve_from_cache(key)    # Cache Read
if value is None:                           # Will enter (1 - SuccessRate) times
    increase_measure("cache_failure")
    with measure_time("cache_operation"):
        value = operation()                 # Main operation to be performed
    with measure_time("cache_write"):
        store_in_cache(key)                 # Cache Write
else:
    increase_measure("cache_success")

What is harder to read.

This will include dashboards and alerts in our monitoring system, to continuously check if it is useful or not anymore.

Conclusions

Sometimes we use a cache system just because we think that caches are fast, but not always are required.

Other times they can be improved by joining operations together. For example, instead of caching separatelly two queries to a database we could join them together in a function and cache the function call, using the function arguments as part of the cache key. This could increase the \(Operation\) time, increasing the Expected improvement percentage, but perhaps could affect to the Success Ratio (\(S_R\)) if it makes it harder to find cached. Once I saw to cache the whole resultant HTML, what is optimous since it required very few parameters (just the user), was retrieved a lot of times and required a lot of queries.

IMHO, using a cache when the Expected improvement percentage is lower than 30% should be discouraged, 50% if cache invalidation is going to be required. This is my feeling because it doesn’t worth the maintenance costs, the code changes, the operation to manage it and, in general, the headhaches it might show.

So answering again…

Should I use a Cache to solve my current problem?

Yes, but only when it will make a difference and is correctly monitored.