Caching — when it helps, when it kills you

Opening

Caching feels like a cheat code. You fetch something expensive — a network response, a database query, a decoded image — and you stash it somewhere fast so you do not have to pay that cost again. Sounds great, right? And it is great. Until it is not. Until your cache serves stale data and your user sees yesterday's profile picture. Until your cache eats so much memory that the system kills your app. Until you spend a week debugging an issue that turns out to be a cached value that should have been invalidated three API calls ago. There is a famous saying in computer science: there are only two hard things — cache invalidation and naming things. Today we are going to understand why caching is both incredibly useful and incredibly dangerous. We will look at how caches actually work, how Android's image loading libraries manage theirs, and how to choose the right cache strategy without shooting yourself in the foot.

What a Cache Actually Is

At its core, a cache is a trade: you spend memory to save time. That is the fundamental bargain. Instead of doing an expensive operation every time — hitting the network, reading from disk, decoding a bitmap — you do it once, store the result in faster storage, and serve from that storage on subsequent requests. The key insight is that caches exploit a pattern called temporal locality: if you accessed something recently, you are likely to access it again soon. Your user scrolled past an image, they will probably scroll back to it. Your app fetched the user's profile, it will need it again in thirty seconds. The weather data you just downloaded is probably still valid for the next ten minutes. A cache bets on this pattern. When the bet pays off — a cache hit — you get massive speedups. When the bet fails — a cache miss — you pay the original cost plus a small overhead for checking the cache. The key to good caching is making sure the bet pays off far more often than it fails.

LRU Cache: The Workhorse

The most common cache strategy you will encounter is LRU — Least Recently Used. The idea is simple. You set a maximum size for the cache. When the cache is full and you need to add a new entry, you evict the entry that was accessed least recently. The assumption is that the thing you used most recently is the thing you are most likely to need again, and the thing you have not touched in the longest time is the safest to throw away. Android provides LruCache right in the framework. It is a LinkedHashMap under the hood, where the ordering tracks access time. When you call get(), the entry moves to the head of the list. When the cache exceeds its max size, entries are removed from the tail. It is elegant and it works surprisingly well for most use cases. But LRU is not magic. It fails badly with scan patterns — if you iterate through a huge list of items once, you will evict your most valuable cached items and replace them with things you will never access again. Keep that in mind.

In-Memory vs Disk Cache

Caches live at different levels, and each level has different trade-offs. An in-memory cache — like LruCache — is blazing fast. Access time is nanoseconds. But it is volatile. When your process dies, it is gone. And it competes with your app's heap for memory. A disk cache — like DiskLruCache — is persistent. It survives process death, app restarts, even device reboots. But it is orders of magnitude slower than memory — milliseconds instead of nanoseconds. The best caching systems use both layers. Check memory first — instant hit if it is there. If not, check disk — slower, but still much faster than network. If not there either, go to the network, and write the result into both caches. This is exactly what image loading libraries like Coil and Glide do. They maintain a two-level cache: a memory cache for images currently or recently on screen, and a disk cache for images that have been downloaded but might not be in memory anymore. The memory cache gives you buttery smooth scrolling. The disk cache means you do not re-download images every time you open the app.

Cache Invalidation: The Hardest Problem

Now we arrive at the truly hard part. How do you know when cached data is no longer valid? This is cache invalidation, and it is genuinely one of the hardest problems in software engineering. Think about it. You cached the user's profile. They just changed their name on another device. Your cache still says the old name. How long is that acceptable? What if you cached a list of products. One was just marked out of stock. Your cache still shows it as available. A user tries to buy it. Cache invalidation strategies come in several flavors. Time-based: the cache entry expires after a fixed duration — say, five minutes. Simple but imprecise. You might serve stale data for four minutes and fifty-nine seconds, or invalidate perfectly fresh data. Event-based: you invalidate when you know something changed — the user just edited their profile, so clear the profile cache. More precise but requires knowing all the events that affect your data. Version-based: each entry has a version or ETag. Before serving, you check if the version is still current. Precise but requires a check on every access. There is no perfect strategy. Every approach involves trade-offs between staleness, complexity, and performance.

Android Image Caching: Coil and Glide Internals

Let us look at how the real-world image loading libraries handle this, because they are masterclasses in caching done right. Coil — Kotlin-first, coroutine-based — uses a two-level cache. The memory cache is an LRU cache keyed by the image URL plus any transformations — like size and crop mode. The key is critical: the same URL at different sizes produces different cache entries. The disk cache stores the raw downloaded bytes. When you load an image, Coil checks memory first. Hit? Return immediately — the bitmap is already decoded. Miss? Check disk. Hit? Decode the bytes into a bitmap, store in memory, return. Miss? Download from network, store raw bytes on disk, decode, store bitmap in memory, return. Glide works similarly but adds a clever twist: it caches at multiple granularities. It caches the original source, the decoded bitmap, and the transformed bitmap separately. So if you apply the same transformation to the same image in two different places, the transformation result is cached, not just the raw download. Both libraries handle invalidation primarily through URL-based keying. Change the URL, and it is a cache miss — fresh download. For time-based invalidation, they rely on HTTP cache headers from the server.

When Cache HURTS

Now let us talk about when caching goes wrong. First: memory pressure. Every byte in your cache is a byte not available for your app's actual work. On Android, your heap is limited — often 256 megabytes or less. A generous image cache can eat 50 to 100 megabytes. That leaves less room for your view hierarchy, your data models, your coroutine stacks. If the system needs memory, it will kill your app. Your cache made the problem worse, not better. Second: stale state. In state-heavy apps — think financial apps, messaging apps, collaborative editing — serving stale data is not just annoying, it is dangerous. If your cache says the account balance is one thousand dollars but the real balance is fifty, that is a serious problem. Third: complexity. Every cache you add is a component that can fail, that needs configuration, that interacts with other systems. Cache-related bugs are notoriously hard to reproduce because they depend on timing, ordering, and the specific contents of the cache at that moment. I have seen teams spend days debugging issues that boiled down to: the cache was serving an old value.

Cache Size: How to Choose

Choosing the right cache size is more art than science, but there are principles. For in-memory image caches, a common starting point is one eighth of your available heap. If your max heap is 256 megabytes, your image cache might be 32 megabytes. Coil and Glide use similar heuristics by default. For disk caches, you have more room — 50 to 250 megabytes is typical for image disk caches. DiskLruCache handles the eviction for you. For data caches — API responses, database query results — the size depends entirely on your data. A cache of user profiles might only need a few hundred entries. A cache of search results might need thousands. The key principle is: measure. Do not guess. Use Android Studio's memory profiler to see how much memory your cache actually consumes. Monitor your cache hit rate — if it is below 70 percent, your cache might be too small, or your access patterns might not benefit from caching at all. If your cache hit rate is above 95 percent and your cache is barely half full, you might be over-allocating.

DiskLruCache and OkHttp Cache

Two important disk caching systems you should understand. DiskLruCache is the standard implementation used by most Android caching libraries. It stores entries as files on disk with a journal file that tracks the LRU ordering. The journal is the clever part — it is an append-only log of cache operations, so tracking access order is cheap. On startup, it replays the journal to reconstruct the LRU state. OkHttp's cache is built on top of DiskLruCache and implements HTTP caching semantics automatically. When a server returns cache headers — Cache-Control max-age, ETag, Last-Modified — OkHttp stores the response on disk. Subsequent requests check the cache. If the cached response is still fresh according to the headers, it is served directly — no network call at all. If it is stale, OkHttp sends a conditional request with If-None-Match or If-Modified-Since. If the server responds with 304 Not Modified, the cached version is still good. This is beautiful because the server controls the invalidation policy. You do not have to guess when data expires — the server tells you.

Practical Cache Patterns

Let me give you some patterns that work well in production Android apps. First: the repository pattern with caching. Your repository checks the cache before hitting the network. If the cache has fresh data, return it immediately. If the cache is stale or empty, fetch from network, update the cache, return. Optionally, return the stale cached value immediately for instant UI display, then refresh in the background and update the UI when fresh data arrives. This is sometimes called stale-while-revalidate, and it gives you both speed and freshness. Second: cache warming. When the user logs in, pre-fetch and cache the data you know they will need — their profile, their recent items, their notification count. The first screen loads instantly. Third: cache scoping. Not everything should live in the same cache. Authentication tokens go in encrypted storage, not LruCache. Image data goes in a memory plus disk cache. API responses might go in a short-lived in-memory cache. Scope your caches to match the lifetime and sensitivity of the data.

When NOT to Cache

Finally, let me be direct about when you should not cache at all. Do not cache data that changes on every request — real-time prices, live scores, streaming data. The cache hit rate will be near zero and you are just wasting memory. Do not cache security-sensitive data in easily accessible caches — tokens, passwords, personal data should go in EncryptedSharedPreferences or the Keystore, not in an LRU cache that could be dumped in a heap analysis. Do not add a cache to fix a performance problem you have not measured. I have seen developers add caching layers speculatively, creating complex invalidation problems, when the actual bottleneck was something completely different — like a missing database index or an unoptimized query. Profile first. Then cache if — and only if — the data shows that repeated expensive lookups are the bottleneck.

Recap and What is Next

Here is what we covered. A cache trades memory for speed. LRU is the most common eviction strategy. Two-level caches — memory plus disk — give you the best of both worlds. Cache invalidation is genuinely hard, with time-based, event-based, and version-based strategies each having trade-offs. Coil and Glide implement sophisticated multi-level caching for images. Caching can hurt through memory pressure, stale data, and added complexity. Size your caches based on measurement, not guessing. OkHttp's cache implements HTTP semantics automatically. And sometimes the best cache is no cache at all. In the next video, we are going to look at something every app does constantly but few developers think about — the hidden cost of strings, buffers, and copies. See you there.