04 · Descent

The chain

Two knobs, one loss. The chain rule splits the blame: how much did w₁ move the hidden, how much did the hidden move the output. Backprop is that, at scale.

In English

Real models have many knobs in a chain: this knob changes a hidden value, that hidden value changes the output, the output changes the loss. The chain rule splits the blame so each knob knows its share. Backprop is that split, automated.

Try this: Nudge each weight. See which one the loss cares about more. That is the gradient flowing backward.
Keep this: Backprop is the chain rule assigning blame to each knob.

Lab · two knobs, one blame

y = w₂ · ReLU(w₁ · x), x=1.4, target=1. Slide either weight. The two gradients say who should move.

w₁ = 0.4

w₂ = 0.8

ŷ / loss: 0.45 · 0.3
∂L/∂w₁ · ∂L/∂w₂: -1.24 · -0.62

∂L/∂w₁ = (∂L/∂ŷ)(∂ŷ/∂h)(∂h/∂w₁). That product is backprop. ReLU zeros the last term on the negative side, which is a dead ReLU.

A transformer is this chain with residual adds (so the gradient has a highway) and layer norms (so the scale stays polite).