Nib asked for
Numbers, Not Letters
When you type a word, the model never receives letters; it receives a list of numbers that stand for pieces of text.
5 steps, drawn by Nib. Read it here, or watch him draw it. Updated .
1what you see
You type 'strawberry' and see ten letters on your screen. You naturally assume the chatbot sees the same thing. It doesn't, and that gap is the whole reason letter-counting goes wrong.
What you see on screen is not what the model receives.
a common mix-upThe chatbot reads text the way a person does, letter by letter.
Do you think the model receives the word as ten separate letters?
Show the answer
No. As the next steps show, it receives numbers standing for pieces of text, not individual letters.
2into numbers
A language model is a giant piece of arithmetic, and arithmetic only works on numbers. Before your message reaches the model, it is converted into a list of numbers. Each number is a label for one piece of your text.
The model only works with numbers, so your text is converted first.
Why must the text be converted into numbers at all?
Show the answer
Because the model is built from arithmetic, which can only operate on numbers, not on letters.
3the catalog
Think of a huge numbered catalog of text pieces, like a phone book where every entry is a chunk of text and has an ID number. Your message is looked up in this catalog, and what the model receives is just the ID numbers in order. The numbers here are illustrative, since real catalogs assign their own.
The model gets ID numbers from a catalog of text pieces, not the text itself.
a common mix-upThe numbers encode the letters in some readable way, like a cipher the model can decode back into spelling.
4sealed boxes
An ID number carries no letters inside it. The number for one chunk tells the model nothing directly about which letters that chunk contains, just as a phone extension doesn't tell you the spelling of the person's name. The model has to learn everything it knows about a chunk from how that chunk behaves in text.
A number is a label, not a container of letters.
a common mix-upThe model can look inside a token to check its letters.
If the model is handed the number for a chunk, does that number tell it how many r's the chunk contains?
Show the answer
No. The number is just a label; any letter-level facts must be learned indirectly.
5bounced off
This is the starting point for the whole puzzle. When you ask 'how many r's are in strawberry?', you are asking about letters, but the model was handed numbers. Asking it to count letters is like asking someone to count the vowels in a word they've only ever heard as a sound label. The next pages show where those chunks come from and why strawberry's letters stay hidden.
A letter-counting question asks about something the model was never directly given.
Remember this: the chatbot never saw your letters, only numbers standing for pieces of text, and that is where the strawberry trouble begins.