Nib asked, and Nib — askNib's tutor — drew the answer live at a whiteboard. This is the spoken transcript; enable JavaScript to watch it drawn.
How do AI language models like ChatGPT actually work? (for a curious adult who uses chatbots but has never studied computing). Cover, in this order: How does ChatGPT work?; How does it read your words?; How does it learn which words come next?; How does it know which earlier words matter?; Why do AI chatbots hallucinate?; Why does it forget what you said earlier?; How did a text predictor learn to follow instructions?; Does it understand, and how much should you trust it? — page 3: Learning From Mountains of Text
We know the machine guesses the next token. But who taught it to guess well? Nobody wrote rules. It learned by reading, a staggering amount of text.
Take a real sentence and hide its next word. The model guesses it, then the original text shows the right answer, so every sentence grades itself.
Inside the model are billions of adjustable numbers, called parameters. Picture knobs on a giant mixing board. At the start they're random, so guesses are gibberish.
After each guess, the model compares its answer with the real one. If it gave the right answer low odds, every knob gets a small push that raises those odds.
Each nudge teaches a little. Repeat it enough times and the model learns spelling, then grammar, then facts and style, since all of them help predict the next word.
Training stops and the knobs lock in place. The text is discarded; what remains is only the stored settings, not a copy of what it read.
So a chatbot's skill is not programmed or looked up: it is the leftover shape of billions of tiny nudges from guessing, checking, and adjusting, over and over.