techtree / Linear algebra / ← prev 4 of 26 next →

Length, angle, and cosine similarity

Length comes from the dot product with itself; dividing out lengths leaves the angle.

46 seconds

Transcript

Take the dot product of a vector with itself. Three times three, plus four times four, is twenty five.

Its square root, five, is the length of the arrow.

It is the long side of this right triangle.

Divide a vector by its length, and it shrinks to length one. Only its direction is left.

Now take two vectors. Shrink both to length one, then take their dot product.

That number is the cosine similarity. It depends only on the angle between them.

Same direction gives one. A right angle gives zero. Opposite gives minus one.

Make one arrow longer, and the number does not change.

Cosine similarity compares directions, and ignores length.

Needs

  • Dot product video
    Multiply matching entries and add them up; one number that measures agreement.
  • Adding and scaling vectors video
    Add vectors entry by entry (tip to tail); scale by multiplying every entry.
Everything this builds on (3 topics, in order)
  1. Vectors
  2. Adding and scaling vectors
  3. Dot product

Leads to

  • Embeddings video
    Each token ID is looked up as a vector; similar meanings point in similar directions.