Engineering Fluency OS

Why This Matters

Every modern AI system — image classifiers, language models, self-driving cars — is built on neural networks. A neural network is a mathematical function that learns patterns from data. It starts knowing nothing, then gradually adjusts millions of internal numbers until it can make accurate predictions.

Understanding how neural networks work gives you the foundation for everything else in deep learning. Without this, transformers, embeddings, and LLMs are black boxes. With it, you can reason about why models behave the way they do and what their limits are.

Define Terms

Visual Model

Input Layer3 features

Hidden Layer4 neurons + ReLU

Output Layer2 predictions

The full process at a glance. Click Start tour to walk through each step.

Data flows forward through layers of neurons, each applying weights, bias, and an activation function.

Code Example

Code

// A single neuron (perceptron) from scratch
function neuron(inputs, weights, bias) {
  // Step 1: Weighted sum
  let sum = bias;
  for (let i = 0; i < inputs.length; i++) {
    sum += inputs[i] * weights[i];
  }
  // Step 2: ReLU activation
  return Math.max(0, sum);
}

// Example: 3 inputs, 3 weights, 1 bias
const inputs = [1.0, 2.0, 3.0];
const weights = [0.2, 0.8, -0.5];
const bias = 0.1;
console.log(neuron(inputs, weights, bias)); // 0.4

// Sigmoid activation function
function sigmoid(z) {
  return 1 / (1 + Math.exp(-z));
}
console.log(sigmoid(0));    // 0.5
console.log(sigmoid(2));    // ~0.88
console.log(sigmoid(-2));   // ~0.12

// Simple 2-layer forward pass
function forwardPass(x, w1, b1, w2, b2) {
  // Hidden layer
  const hidden = w1.map((row, i) =>
    Math.max(0, row.reduce((s, w, j) => s + w * x[j], 0) + b1[i])
  );
  // Output layer
  const output = w2.map((row, i) =>
    row.reduce((s, w, j) => s + w * hidden[j], 0) + b2[i]
  );
  return output;
}

console.log("Forward pass:", forwardPass(
  [1, 0],
  [[0.5, -0.3], [0.8, 0.1]],
  [0.1, -0.2],
  [[0.6, 0.4]],
  [0.0]
));

Interactive Experiment

Try these exercises:

Change the weights in the single neuron example to all zeros. What does the neuron output? Why?
Replace ReLU with sigmoid in the neuron function. How does the output range change?
Add a third hidden neuron to the forward pass. What changes in the weight matrices?
Set the bias to a very large negative number. What happens to ReLU output? (This is a "dead neuron.")

Quick Quiz

Coding Challenge

Build a Multi-Neuron Layer

Write a function called `denseLayer` that takes an array of inputs, a 2D array of weights (one row per neuron), an array of biases (one per neuron), and applies ReLU activation. Return an array of output values, one per neuron.

Loading editor...

Real-World Usage

Neural networks power the most impactful technologies of the modern era:

Image recognition: Convolutional neural networks identify objects in photos, enable face unlock, and power medical imaging diagnosis.
Language models: GPT, Claude, and other LLMs are massive neural networks trained on text to predict the next token.
Recommendation systems: Netflix, Spotify, and YouTube use neural networks to predict what you want to watch or listen to next.
Game AI: AlphaGo and AlphaFold use deep neural networks to master Go and predict protein structures.
Self-driving cars: Neural networks process camera and lidar data to detect lanes, obstacles, and traffic signs in real time.

Neural Networks