01 · The score

Loss

Loss is a number that should get smaller. Squared error is the toy. Cross-entropy is the one LLMs actually minimize — Volume VI.

In English

Loss is a number that should get smaller. “How wrong is this guess?” Squared error is the school version (miss by 2, pay 4). Language models use a cousin called cross-entropy — same job, different formula.

Try this: Move the guess away from the target. The score should climb. That climb is what training fights.
Keep this: Loss is a score of wrongness. Training is making it smaller.

Lab · miss the target

True value is 1. Drag the prediction. Loss is (ŷ − y)².

Prediction ŷ = 0.2

Loss 0.64

A loss is a scalar you can differentiate. Squared error is convex here, so one bottom. Classification uses cross-entropy instead — Volume VI.

Training is not “make the model smarter.” It is “make this number smaller on the data you have, without ruining the data you do not.”