Nib asked, and Nib — askNib's tutor — drew the answer live at a whiteboard. This is the spoken transcript; enable JavaScript to watch it drawn.
how does backpropagation work in neural networks
A tiny network takes x equals 2 and guesses 3, but the right answer is 1. The loss, the squared error, is 4. Two weights sit in the chain.
Which weight deserves the blame, and how much? Backpropagation computes that for every weight at once. It only finds the blame; it changes nothing itself.
Now go backward, from the loss. The loss is the guess minus the target, squared, so its slope with respect to y is 2 times 3 minus 1, which is 4.
Chain rule: multiply slopes as you walk back. Going through w2, h gets 4 times 0.5, which is 2. And w2 itself gets 4 times h, which is 24.
One more step back. The slope at h is 2, and h came from x times w1, so w1 gets 2 times x, which is 2 times 2, which is 4.
Now gradient descent finally does the learning: each weight moves against its slope. With a rate of 0.01, w1 becomes 2.96 and w2 becomes 0.26, and the loss drops from 4 to about 0.29.
Backpropagation is just the chain rule run backward, handing every weight its share of the blame in a single pass.