Engineering Fluency OS

Why This Matters

Once you know how to compute partial derivatives, a natural question arises: what happens when you do not move along a coordinate axis but instead move in a diagonal or arbitrary direction? The directional derivative answers that question. It tells you the rate of change of a function in any direction you choose, not just along x or y.

The gradient vector bundles all the partial derivatives together into one object. It has a remarkable property: it always points in the direction of steepest ascent. This single fact powers gradient descent, the workhorse algorithm behind modern machine learning. To minimize a loss function, you simply walk opposite to the gradient. Understanding the gradient also gives you level curves, tangent planes, and the geometry of multivariable optimization for free.

Define Terms

Visual Model

f(x, y)Scalar function

df/dxPartial w.r.t. x

df/dyPartial w.r.t. y

Gradient(df/dx, df/dy)

Unit Vector uDirection of travel

D_u fDirectional derivative

Steepest Ascentu = grad / |grad|

The full process at a glance. Click Start tour to walk through each step.

The gradient vector bundles partial derivatives and points toward steepest ascent. The directional derivative is its dot product with any unit direction.

Code Example

Code

// Gradient and directional derivative
// f(x, y) = x^2 + x*y
function f(x, y) {
  return x * x + x * y;
}

function gradient(fn, x, y, h = 1e-6) {
  const dfdx = (fn(x + h, y) - fn(x - h, y)) / (2 * h);
  const dfdy = (fn(x, y + h) - fn(x, y - h)) / (2 * h);
  return [dfdx, dfdy];
}

function dot(a, b) {
  return a[0] * b[0] + a[1] * b[1];
}

function magnitude(v) {
  return Math.sqrt(v[0] * v[0] + v[1] * v[1]);
}

function normalize(v) {
  const mag = magnitude(v);
  return [v[0] / mag, v[1] / mag];
}

// Gradient at (1, 3)
const grad = gradient(f, 1, 3);
console.log("Gradient at (1,3):", grad.map(v => v.toFixed(4))); // [5, 1]

// Directional derivative in direction (1, 1) normalized
const u = normalize([1, 1]);
const dirDeriv = dot(grad, u);
console.log("Direction:", u.map(v => v.toFixed(4)));
console.log("Directional derivative:", dirDeriv.toFixed(4)); // ~4.2426

// Steepest ascent direction
const steepest = normalize(grad);
console.log("Steepest ascent direction:", steepest.map(v => v.toFixed(4)));
console.log("Max rate of change:", magnitude(grad).toFixed(4)); // ~5.0990

Interactive Experiment

Try these exercises:

Compute the gradient of f(x, y) = x^2 + y^2 at several points. Notice the gradient always points radially outward from the origin. Why?
Pick the unit vector pointing in the negative gradient direction and compute the directional derivative. Verify it equals the negative of the gradient magnitude.
For f(x, y) = x*y, find a direction at (1, 1) where the directional derivative is zero. What does that direction correspond to geometrically?
Compute the gradient of f(x, y) = sin(x)*cos(y) at (pi/4, pi/4). Which direction gives the steepest increase?
Try gradient descent: start at (3, 4) on f(x,y) = x^2 + y^2, repeatedly move in the negative gradient direction with step size 0.1. How quickly do you approach (0, 0)?

Quick Quiz

Coding Challenge

Gradient and Directional Derivative

Write a function called `gradientAndDir` that takes x, y, dx, dy as inputs. Compute the numerical gradient of f(x, y) = x^2*y - y^3 at (x, y) using central differences with h = 1e-5. Then compute the directional derivative in the direction (dx, dy) (normalize it to a unit vector first). Return a string: `gx,gy,dd` where gx and gy are the gradient components and dd is the directional derivative, all rounded to 4 decimal places.

Loading editor...

Real-World Usage

Gradients and directional derivatives power critical systems:

Machine learning: Every training step in neural networks computes the gradient of the loss function with respect to millions of parameters and takes a step opposite to it (gradient descent).
Image processing: Edge detection filters (Sobel, Canny) compute the gradient of image intensity to find boundaries between regions.
Meteorology: Weather maps show pressure gradients. Wind flows from high to low pressure, approximately in the negative gradient direction (adjusted for Coriolis force).
Robotics: Path planning uses gradient fields to guide robots away from obstacles and toward goals.
Finance: Sensitivity analysis computes the gradient of portfolio value with respect to market parameters (the "Greeks" in options pricing are partial derivatives).

Gradients and Directional Derivatives