techtree
A map of what you need to know to understand a transformer, the machine inside modern AI. Each topic gets one short video and builds only on the topics before it. The numbers are a watch order: every topic comes after everything it needs.
Assumes: arithmetic, basic algebra, functions and their graphs, exponents, square roots. 26 of 26 videos made so far.
Linear algebra
- 1Vectors video
A vector is a list of numbers, and also an arrow. - 2Adding and scaling vectors video
Add vectors entry by entry (tip to tail); scale by multiplying every entry. - 3Dot product video
Multiply matching entries and add them up; one number that measures agreement. - 4Length, angle, and cosine similarity video
Length comes from the dot product with itself; dividing out lengths leaves the angle. - 5Matrices video
A matrix is a grid of numbers; a stack of vectors. - 6Matrix times vector video
Each output entry is a dot product of one row with the vector. - 7Matrix times matrix video
Multiply a matrix by many vectors at once. - 8Transpose video
Flip a matrix so rows become columns.
Glue math
- 9Exp and softmax video
Turn any list of scores into positive weights that add up to one. - 10Nonlinearity (ReLU) video
Clip negatives to zero, entry by entry; the bend that makes layers useful. - 11Mean, variance, and normalization video
Shift and scale a vector so its entries have mean 0 and spread 1.
Building blocks
- 12Tokens video
Text is chopped into pieces, and each piece gets an ID number. - 13Embeddings video
Each token ID is looked up as a vector; similar meanings point in similar directions. - 14Position information video
Add a position vector to each token vector so order is not lost. - 15A neural network layer video
Multiply by a matrix, add a bias, apply a nonlinearity. - 16MLP video
Two layers in a row; widen, bend, narrow. - 17Residual connections video
Add a block's output back to its input, so each block makes an edit.
Attention
- 18Queries, keys, and values video
Three matrices turn each token vector into a query, a key, and a value. - 19Attention scores video
Dot each query with every key, then softmax into weights. - 20Self-attention video
Each token's output is a weighted average of the value vectors. - 21Causal masking video
Tokens may only look backward, never at the future. - 22Multi-head attention video
Run several small attentions side by side and combine them.
Transformer
- 23The transformer block video
Attention then MLP, each wrapped in normalization and a residual. - 24Stacking blocks video
The same block design, repeated many times, each with its own weights. - 25Next-token prediction and sampling video
Turn the last vector into a score per token, softmax, and pick one. - 26What is a transformer? video
The whole machine, end to end.