Backpropagation: The Chain Rule and How Gradients Flow Backward
~15 min read
Backpropagation is just the chain rule from calculus, applied layer by layer in reverse — it tells every weight in the network exactly how much it contributed to the final error, so it knows which way to adjust.
Backpropagation: The Chain Rule and How Gradients Flow Backward is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Backpropagation computes how much each weight, anywhere in the network, contributed to the final loss — using the chain rule from calculus
- •It works BACKWARD from the loss toward the input, one layer at a time, because each layer's gradient depends on the gradient from the layer after it
- •Each layer only needs the gradient handed to it from the next layer plus its own local derivative — this locality is what makes it efficient
- •The output is a gradient for every weight: the direction that would make the loss WORSE if you moved that weight that way
- •Weights are then updated in the OPPOSITE direction of their gradient (gradient descent) — backprop finds the gradients, gradient descent uses them