06 · Invariants

Projection

Drop a perpendicular onto a line. Attention is this, with weights: how much of this vector sits along that one. A residual is what is left.

In English

Drop a perpendicular onto a line: that is the shadow of your arrow on that line. Attention is this idea with weights — “how much of you sits along me?” What is left over is the residual.

Try this: Drag the vector. Read the shadow on the axis and the leftover. The two pieces should rebuild the original.
Keep this: Projection is the shadow. Attention is a learned, weighted shadow.

Lab · drop a perpendicular

Drag v and the axis. The solid arrow on the line is the projection. The leftover is the residual.

Weight ⟨axis|v⟩: 1.8
Projection: [1.77, 0.35]

proj = ⟨axis, v⟩ axiŝ. The residual is v minus that. Orthogonal means the leftover is perpendicular to the axis you asked about.

Attention: Q names the axis, K scores candidates, V is what gets added. Residual connections keep the leftover on purpose.