Volume V · Learning學習
The model is a fitted function. Loss, a slope, a step, then the ways that fails.
A language model is not a mind. It is a function with a lot of knobs, fitted so next-token surprise is small. This volume is those knobs moving.
In English
A model is a function with knobs. Training turns the knobs so the function is less wrong on examples. This volume is the score, the slope, the step, and the ways that fails.
- Try this
- Start at Loss. If “gradient” is a blank, stay in order.
- Keep this
- Learning is: measure wrongness, see the slope, step downhill.
The stack · click a layer
What this is
A one-weight bowl, a two-weight chain, a polynomial that memorizes, a saddle. Enough to see what “training” is before you trust a 400B-parameter slide.
What this is not
Not PyTorch and not a full backprop engine. Squared error and a ReLU chain are the toys. The LLM objective is cross-entropy, in Volume VI.
- 01 · The scoreLoss損失Miss the target. Watch the score.Loss is a score of wrongness. Training is making it smaller.
- 02 · The slopeGradient梯度Slide a weight. Read the slope.Gradient = steepest increase. We step the opposite way.
- 03 · DescentThe step步進Pick a step size. Fall downhill.$w \leftarrow w - \eta \times$ slope. $\eta$ is a taste, not a law.
- 04 · DescentThe chain鏈式Two weights. See who gets the gradient.Backprop is the chain rule assigning blame to each knob.
- 05 · CapacityOverfit過擬合Raise the degree. Fit the noise.Fitting the points is not the same as fitting the function.
- 06 · CapacityLandscape景觀Walk a saddle. Walk a well.Real loss is not a bowl. SGD wanders saddles anyway.