Engineering Fluency OS

Why This Matters

A database query takes 50 milliseconds. A cache lookup takes 1 millisecond. If your application serves 10,000 requests per second for the same data, the choice between querying the database every time and caching the result is the difference between a fast application and a crashed one. Caching is the single most impactful performance optimization in system design.

Caches work by storing the result of an expensive operation (a database query, an API call, a complex computation) so that future requests for the same data can be served instantly. But caching introduces its own challenges: stale data, cache invalidation, memory limits, and consistency. Understanding when and how to cache -- and when NOT to cache -- is a core skill. Every major website you use, from Google to Netflix, relies heavily on caching at multiple levels.

Define Terms

Visual Model

ClientSends request

CacheFast lookup (O(1))

Cache HitReturn immediately

DatabaseSlow query

Store ResultPopulate cache

Lookup

Hit

Miss

Write

Response

The full process at a glance. Click Start tour to walk through each step.

The cache check pattern: hit the cache first, fall back to expensive computation on a miss, then populate the cache.

Code Example

Code

// In-memory cache with TTL (Time to Live)
class Cache {
  constructor(ttlMs = 60000) {
    this.store = new Map();
    this.ttl = ttlMs;
  }

  get(key) {
    const entry = this.store.get(key);
    if (!entry) return null; // cache miss
    if (Date.now() > entry.expiry) {
      this.store.delete(key); // expired
      return null; // cache miss
    }
    return entry.value; // cache hit
  }

  set(key, value) {
    this.store.set(key, {
      value,
      expiry: Date.now() + this.ttl,
    });
  }
}

// Usage: cache-aside pattern
const cache = new Cache(30000); // 30 second TTL

async function getUser(userId) {
  // 1. Check cache first
  const cached = cache.get(`user:${userId}`);
  if (cached) {
    console.log("Cache hit!");
    return cached;
  }

  // 2. Cache miss — query database
  console.log("Cache miss — querying DB");
  const user = await db.query(
    "SELECT * FROM users WHERE id = $1", [userId]
  );

  // 3. Store in cache for next time
  cache.set(`user:${userId}`, user);
  return user;
}

Interactive Experiment

Try these exercises:

Implement the Cache class above and call getUser twice with the same ID. Observe "Cache hit!" on the second call.
Set the TTL to 2 seconds. Call getUser, wait 3 seconds, call it again. Is the second call a hit or miss?
What happens if you update a user in the database but the old data is still cached? How would you solve this?
Add a delete(key) method to the Cache class. When would you use it?

Quick Quiz

Coding Challenge

LRU Cache

Write a `LRUCache` class (Least Recently Used) with a maximum capacity. It should have `get(key)` and `set(key, value)` methods. When the cache is full and a new key is added, evict the least recently used entry. Both `get` and `set` should mark the key as recently used. `get` returns null if the key is not found.

Loading editor...

Real-World Usage

Caching is fundamental to every high-performance system:

Redis and Memcached: In-memory caching servers used by almost every major web application to cache database results, sessions, and API responses.
CDNs (CloudFront, Cloudflare): Cache static assets (images, CSS, JS) at edge locations worldwide, reducing page load times from seconds to milliseconds.
Browser cache: Your browser caches CSS, JavaScript, images, and API responses locally to avoid re-downloading them on every page load.
Database query cache: Databases like MySQL and PostgreSQL cache the results of frequently run queries.
CPU cache (L1, L2, L3): Even at the hardware level, CPUs cache recently accessed memory to avoid slow main memory reads.

Caching Strategies