Vectors & Vector Spaces

Why This Matters

A vector is one of the most versatile objects in all of mathematics and computer science. At its simplest, a vector is an ordered list of numbers, but that humble definition unlocks everything from 3D game physics to machine learning embeddings to search engines. Whenever you represent a data point as a list of features, you are working with vectors. Whenever a neural network processes an input, it is multiplying vectors by matrices millions of times per second.

A vector space is the mathematical arena where vectors live. It defines the rules for adding vectors together and scaling them, and those rules guarantee that the results stay inside the space. Understanding vector spaces gives you the language to reason about dimensionality, independence, and basis -- the core concepts that power data science and computer graphics.

Linear combinations tie it all together. When you scale some vectors and add the results, you get a linear combination. The set of all possible linear combinations of a group of vectors is called their span. Asking whether a vector can be written as a linear combination of others is the fundamental question of linear algebra, and it shows up everywhere from solving equations to compressing data.

Define Terms

Visual Model

Code Example

// Vector operations in JavaScript

// Represent vectors as arrays
const v1 = [3, 4];
const v2 = [1, -2];

// Vector addition: component-wise
function vecAdd(a, b) {
  return a.map((val, i) => val + b[i]);
}
console.log("v1 + v2:", vecAdd(v1, v2)); // [4, 2]

// Scalar multiplication
function scalarMul(c, v) {
  return v.map(val => c * val);
}
console.log("3 * v1:", scalarMul(3, v1)); // [9, 12]

// Linear combination: c1*v1 + c2*v2
function linearCombination(scalars, vectors) {
  const dim = vectors[0].length;
  const result = new Array(dim).fill(0);
  for (let i = 0; i < vectors.length; i++) {
    for (let j = 0; j < dim; j++) {
      result[j] += scalars[i] * vectors[j][j];
    }
  }
  return result;
}

// Simpler: combine two vectors
function linComb2(c1, v1, c2, v2) {
  return vecAdd(scalarMul(c1, v1), scalarMul(c2, v2));
}
console.log("2*v1 + 3*v2:", linComb2(2, v1, 3, v2)); // [9, 2]

// Magnitude (length) of a vector
function magnitude(v) {
  return Math.sqrt(v.reduce((sum, x) => sum + x * x, 0));
}
console.log("magnitude of [3,4]:", magnitude(v1)); // 5

// Check: is [9, 2] a linear combination of v1 and v2?
// We need c1*3 + c2*1 = 9 and c1*4 + c2*(-2) = 2
// Solving: c1 = 2, c2 = 3. Yes!
console.log("Verify: 2*v1 + 3*v2 =", linComb2(2, v1, 3, v2));

Interactive Experiment

Try these exercises:

• Add [1, 2, 3] and [4, 5, 6] by hand, then verify with code. What happens if the vectors have different lengths?
• Compute 0.5 * [10, 20]. Then compute -1 * [3, 4]. What does a negative scalar do geometrically?
• Find scalars c1 and c2 such that c1 * [1, 0] + c2 * [0, 1] = [7, 3]. Is this always possible in 2D?
• Try to write [1, 1] as a linear combination of [2, 2] and [4, 4]. Why is this impossible for most target vectors?
• Compute the magnitude of [3, 4] and [5, 12]. Do you recognize these as Pythagorean triples?

Quick Quiz

Coding Challenge

Real-World Usage

Vectors and vector spaces are foundational across computing:

• Machine learning: Every data point is a feature vector. Word embeddings like Word2Vec represent words as 300-dimensional vectors where similar words are close together.
• Computer graphics: Positions, directions, normals, and velocities are all 3D vectors. Every vertex in a 3D model is a vector, and the GPU performs billions of vector operations per frame.
• Search engines: Documents and queries are represented as high-dimensional vectors (TF-IDF or embeddings). Search is finding the closest vectors in that space.
• Physics simulations: Forces, velocities, and accelerations are vectors. Adding force vectors determines net force. Games and simulations rely on this constantly.
• Recommendation systems: Users and items are embedded as vectors. The closer a user vector is to an item vector, the more likely the user is to enjoy that item.

Connections