Nib asked, and Nib — askNib's tutor — drew the answer live at a whiteboard. This is the spoken transcript; enable JavaScript to watch it drawn.
Why can't AI count the r's in strawberry? (for a curious adult who uses chatbots but has never studied computing). Cover, in this order: What does an AI actually receive when you type a word?; Where do tokens come from?; Why is strawberry split into pieces the model can't see inside?; How does a model learn spelling at all?; Why does letting it write the letters out fix the count?; What else do tokens explain? — page 2: Where Tokens Come From
Last page: the model gets numbers for pieces of text. Who picks those pieces? It's decided before training, by a fixed procedure, not by the model.
Start with single characters. Scan lots of text, find the pair that appears side by side most often, glue it into one piece. Repeat, thousands of times.
For example, imagine 'e' and 'r' appear together constantly. They merge into 'er'. Later rounds glue more pairs, building chunks like 'straw'.
Common words get a single token. Rare or long words are split into several smaller pieces, each with its own number.
To type a word, swap each chunk for its number. The letters inside a chunk stay behind; only the number travels with the word.
So a word's pieces are decided by frequency in old text, not by its letters, and next we'll see what that does to strawberry.