Nib asked, and Nib — askNib's tutor — drew the answer live at a whiteboard. This is the spoken transcript; enable JavaScript to watch it drawn.
how does a neural network learn from data
Nobody types the rule into a neural network. It starts with a guess and improves. Here are three examples: an input, and the right answer for it.
A neuron multiplies its input by a number called a weight. Start with a poor guess, 0.5, and the line falls short of every example.
The gap between guess and truth is the error: 1.5, then 3, then 4.5. Add them up and you get one score for how wrong the network is.
Nudge the weight up to 1.2 and the gaps shrink. Keep nudging while they keep shrinking.
So learning is a loop. Guess, measure the error, nudge each weight in the direction that shrinks it, then repeat on the examples, over and over.
A real network has millions of weights, not one. Backpropagation tells each weight which way to nudge, and the same loop runs on all of them.
After many nudges the weight lands near 2, and the line fits every example. Better still, a new input, 4, gets the answer 8. It learned a rule, not a list.
A neural network learns by nudging numbers: guess, check the error, adjust, repeat, until the rule it holds works on new inputs too.