Engineering Fluency OS

Why This Matters

Your database query takes 50 milliseconds. That sounds fast until your API gets 10,000 requests per second. Now you are asking the database to handle 10,000 queries per second -- 500 seconds of work crammed into one second. The database becomes the bottleneck. Caching solves this by storing frequently-accessed results in fast memory so most requests never hit the database at all.

But caching introduces a fundamental problem: cache invalidation. When the underlying data changes, how do you ensure the cache does not serve stale (outdated) data? Phil Karlton famously said there are only two hard things in computer science: cache invalidation and naming things. He was not joking. The cache-aside pattern is the most common strategy, but understanding the tradeoffs is essential.

Define Terms

Visual Model

ApplicationGET /user/42

CacheRedis / Memcached

Cache Hitreturn instantly

Cache Miss

DatabasePostgreSQL

Store + TTLcache for 300s

Return Data

1. check

found

not found

2. query

3. result

4. populate

The full process at a glance. Click Start tour to walk through each step.

Cache-aside: check cache first, query DB on miss, store result with TTL. Invalidate when data changes.

Code Example

Code

-- SQL: No caching concept in SQL itself,
-- but here is what the application layer does:

-- Step 1: Application checks cache (pseudocode)
-- cache_key = "user:42"
-- cached = redis.GET(cache_key)
-- if cached exists, return JSON.parse(cached)

-- Step 2: Cache miss -> query database
SELECT id, name, email FROM users WHERE id = 42;

-- Step 3: Store result in cache with TTL
-- redis.SET(cache_key, JSON.stringify(result), EX, 300)
-- EX 300 = expire after 300 seconds (5 minutes)

-- Step 4: On data change, invalidate cache
UPDATE users SET email = new@test.com WHERE id = 42;
-- Application must also: redis.DEL("user:42")

-- Common cache key patterns:
-- "user:{id}"           -> single user by ID
-- "user:email:{email}"  -> user lookup by email
-- "products:category:electronics:page:1" -> paginated list
-- "stats:daily:2024-01-15" -> aggregated report

-- Cache stampede problem:
-- 1000 requests arrive at the same time
-- Cache key just expired
-- All 1000 hit the database simultaneously!
-- Solution: use a lock so only one request queries DB,
-- others wait for the cache to be repopulated

Interactive Experiment

Try these exercises:

Implement a simple in-memory cache (a dictionary) with TTL. Set values with expiration times and verify they disappear after the TTL.
Simulate the cache-aside pattern: first call misses and "queries the DB" (use a mock function with a delay), second call hits the cache instantly.
Demonstrate cache invalidation: update the "database" value and observe the cache serving stale data until you invalidate it.
Simulate a cache stampede: 100 "requests" arrive at once when the cache is empty. How many hit the "database"? Add a lock so only one does.

Quick Quiz

Coding Challenge

Build a Cache with TTL

Write a class called `TTLCache` that stores key-value pairs with a Time-To-Live (TTL) in seconds. It should support `set(key, value, ttl)`, `get(key)` (returns undefined/None if expired or missing), and `delete(key)`. Use timestamps to check expiration.

Loading editor...

Real-World Usage

Caching is used at every level of modern systems:

Redis: The most popular caching system. Used by Twitter, GitHub, and Stack Overflow to cache user sessions, timelines, and hot data. Sub-millisecond lookups.
CDN caching: Content Delivery Networks like Cloudflare cache static assets and API responses at edge locations worldwide.
Application-level caching: Frameworks like Django (cache framework), Rails (Russian doll caching), and Next.js (ISR) have built-in caching.
Database-level caching: PostgreSQL uses a buffer pool (shared_buffers) to cache frequently-accessed pages in memory.
Connection pooling: Tools like PgBouncer cache database connections so each request does not pay the cost of establishing a new connection.

Database Caching