Opening
In the last video we learned that threads are data structures the kernel manages, and that the scheduler rapidly switches between them to create the illusion of parallelism. But we glossed over something important: what actually happens during that switch? Because it turns out that switching from one thread to another is not free. Not even close. There is a real, measurable cost to every context switch, and if you do not understand that cost, you will write code that looks concurrent but performs terribly. Today we are going to look at context switching under a microscope. We will see exactly what the CPU has to do, why it takes time, and most importantly, why the obvious cost is not even the real problem. The real killer is something you probably have not thought about: cache invalidation.
One Core, Many Threads
Let us start with a fundamental reality. A single CPU core can only execute one thread at a time. One. That is a hardware constraint. It does not matter how many threads you create, a single core runs exactly one of them at any given nanosecond. So when your Android phone has eight cores and two hundred threads across all running apps, those threads are not all running simultaneously. At most eight are truly executing. The rest are waiting. The scheduler creates the illusion of concurrency by rapidly switching which thread each core is running. It gives thread A a few milliseconds, pauses it, switches to thread B, gives it a few milliseconds, pauses it, switches to thread C, and so on. This switching happens so fast, thousands of times per second, that it looks like everything is running at once. But each switch has a price.
Anatomy of a Context Switch
So what exactly happens when the kernel decides to switch from thread A to thread B on a given core? There is a precise sequence of steps. First, something triggers the switch. Maybe thread A's time slice expired. Maybe thread A called a blocking system call like reading from disk. Maybe a higher-priority thread became ready. Whatever the reason, the kernel takes control. Step one: save thread A's state. The kernel copies all of the CPU registers, the program counter, the stack pointer, the status flags, everything, into thread A's thread control block in kernel memory. Step two: update thread A's state to "ready" or "blocked," depending on why the switch happened. Step three: select thread B from the ready queue. The scheduler picks the highest-priority thread that is waiting to run. Step four: load thread B's state. The kernel copies thread B's saved registers, program counter, and stack pointer from its control block back into the actual CPU registers. Step five: jump to thread B's program counter and start executing. From thread B's perspective, it is as if it was never paused. It picks up exactly where it left off.
The Direct Cost
Now, how long does all of this take? The raw mechanics of saving and restoring registers and switching page tables typically take somewhere between one and ten microseconds, depending on the processor and operating system. That might sound insignificant. One microsecond is one millionth of a second. But consider the scale. If your system is doing ten thousand context switches per second, which is quite normal for an active Android phone, that is ten to one hundred milliseconds of pure overhead every second. That is real time being spent on bookkeeping rather than useful work. And on Android, where you have a sixteen millisecond budget to render each frame at sixty frames per second, a few extra context switches at the wrong moment can mean the difference between smooth scrolling and a visible jank. But here is the thing: this direct cost, the register save and restore, is actually the small part. The big cost is what happens after the switch.
The Cache Hierarchy
To understand the real cost, you need to understand CPU caches. Your processor does not read data directly from main RAM every time it needs something. RAM is slow, relatively speaking. Fetching a value from RAM takes around one hundred nanoseconds. So the CPU has small, incredibly fast memory buffers called caches, built right into the chip. There are typically three levels. L1 cache is the fastest and smallest, maybe 32 or 64 kilobytes, with an access time of about one nanosecond. L2 cache is larger, maybe 256 kilobytes to a megabyte, with access times around four to seven nanoseconds. L3 cache is shared across all cores, several megabytes, with access times around ten to twenty nanoseconds. When the CPU needs data, it checks L1 first. If the data is there, great, that is a cache hit. If not, it checks L2, then L3, and finally main RAM. Each miss costs more time. The beautiful thing about caches is that they work because of locality. When your thread is running, it tends to access the same memory regions repeatedly: its stack, its local data, the objects it is working with. Over time, the caches fill up with exactly the data that thread needs, and everything runs at cache speed. Fast. Really fast.
Cache Invalidation: The Real Killer
And now here is the punch line. When you context switch from thread A to thread B, all of that beautiful cache warmth that thread A built up? Thread B does not care about it. Thread B has its own stack, its own data, its own working set. So what happens? Thread B starts executing and immediately starts missing the cache. The data it needs is not in L1, not in L2, maybe not even in L3. Every access goes all the way to main RAM. One hundred nanoseconds instead of one. And it is not just a few accesses. It might take thousands of memory operations before the caches warm up with thread B's data. This phenomenon is called cache pollution or cache thrashing, and it is the dominant cost of context switching. Studies have shown that the effective cost of a context switch, including cache effects, can be twenty to fifty microseconds or more, not the one to ten microseconds of the raw register swap. For workloads that are cache-sensitive, which includes most real-world code, context switches can slow things down by a factor of two to five compared to running a thread uninterrupted.
Why 100 Threads on 8 Cores Is a Problem
Now let us connect this to a concrete scenario. Say you have an Android device with eight cores, and your app creates a hundred threads all doing CPU-intensive work. What happens? At any moment, only eight threads can run. The other ninety-two are waiting. The scheduler is constantly cycling through them, giving each a turn. With a hundred threads competing for eight cores, you get a massive number of context switches. And every switch brings cache invalidation. Your threads are constantly evicting each other's data from the caches, and every time a thread resumes, it spends a significant portion of its time slice just reloading its working data into the cache. The CPU is busy, utilization might look like one hundred percent in a profiler, but a huge fraction of that "busy" time is wasted on cache misses rather than actual computation. This is why creating more threads than cores for CPU-bound work does not make your code faster. It makes it slower. You are not adding parallelism. You are adding overhead.
How Dispatchers.Default Limits Thread Count
The Kotlin coroutines team understood this. That is why Dispatchers.Default creates a thread pool sized to the number of CPU cores. If your phone has eight cores, the default dispatcher has roughly eight threads. When you launch a hundred coroutines on Dispatchers.Default, they do not each get their own thread. Instead, they are queued up and executed on those eight threads, a few at a time. When one coroutine suspends, another one starts on the same thread, without a kernel-level context switch. The thread stays on the core, its cache stays warm, and the switching happens at the coroutine level, which is just shuffling some objects in user space. This is a deliberate design choice to avoid the context switching problem. Dispatchers.IO, by contrast, allows more threads because IO-bound work spends most of its time blocked, so cache warming matters less. But for CPU-bound work, Dispatchers.Default keeps the thread count low on purpose.
How Coroutines Avoid the Cost
Let us be specific about why coroutine switching is cheaper than thread switching. When a coroutine suspends, the kernel is not involved at all. There is no trap into kernel mode. No register save and restore through the OS. No page table manipulation. What happens is pure user-space bookkeeping. The coroutine's state, its local variables and its suspension point, is saved into a small object on the heap, a Continuation. Then the thread's event loop simply picks the next coroutine from its queue and starts running it. Since the same thread is running both coroutines, the L1 and L2 caches are still warm with that thread's data. Yes, the new coroutine might access different heap objects, but the stack remains the same thread's stack, and many of the memory regions overlap. The result is that "switching" between coroutines on the same thread can be one or two orders of magnitude cheaper than a full kernel context switch. This is the secret sauce behind coroutines' performance advantage for concurrent workloads.
Measuring Context Switches
By the way, you can actually measure context switches on your device. If you use Android Studio's profiler and look at the CPU trace, you can see when your threads are running, when they are sleeping, and when they are being switched in and out. On a Linux system, the tool vmstat shows system-wide context switch rates. You can also check per-process stats in the proc filesystem. If you see your app doing thousands of involuntary context switches per second, that is a sign you might have too many active threads competing for CPU time. The fix is usually to reduce the number of threads doing CPU work and use coroutines or a properly sized thread pool to match your core count.
Recap and What is Next
Let us wrap up. A context switch happens when the kernel moves a CPU core from one thread to another. It saves and restores register state, which takes one to ten microseconds. But the real cost is cache invalidation: the new thread's data is not in cache, leading to expensive RAM accesses that can cost twenty to fifty microseconds or more in effective overhead. Too many threads competing for too few cores means excessive switching and cache thrashing. Dispatchers.Default limits its pool to core count to avoid this. Coroutines switch in user space without kernel involvement, keeping caches warm and overhead minimal. In the next video, we are going to look at a different and even more dangerous consequence of threads sharing memory: race conditions. When two threads read and write the same data, things break in ways that are maddeningly hard to debug. See you there.