How does a neural network learn from data?

1the examples

Nobody types the rule into a neural network. It starts with a guess and improves. Here are three examples: an input, and the right answer for it.

2a weak guess

A neuron multiplies its input by a number called a weight. Start with a poor guess, 0.5, and the line falls short of every example.

3the error

The gap between guess and truth is the error: 1.5, then 3, then 4.5. Add them up and you get one score for how wrong the network is.

4a nudge

Nudge the weight up to 1.2 and the gaps shrink. Keep nudging while they keep shrinking.

5the loop

So learning is a loop. Guess, measure the error, nudge each weight in the direction that shrinks it, then repeat on the examples, over and over.

6millions of weights

A real network has millions of weights, not one. Backpropagation tells each weight which way to nudge, and the same loop runs on all of them.

7the whole idea

After many nudges the weight lands near 2, and the line fits every example. Better still, a new input, 4, gets the answer 8. It learned a rule, not a list.