How do AI language models like ChatGPT actually work?

Start with something familiar. Type “see you” into your phone and it suggests “soon”. It does not know your plans. It only knows what usually follows. A chatbot like ChatGPT is that same idea scaled up enormously.

Give it a sentence to finish, such as “The capital of France is”. The model scores every candidate for the next piece of text. “Paris” gets a very high score, a few alternatives get small ones, and “banana” gets almost none. It is not looking the answer up in a database; the score comes from patterns it absorbed in training.

Then comes the trick. The chosen piece is added to the text, the longer text goes back into the model, and it guesses again, round and round, until the reply is finished. Every fluent answer you have ever read from a chatbot was built one guess at a time.

The guess is a choice among likely options rather than one fixed answer. That is why, with the usual settings, asking the same question twice can produce different wording, and why early choices steer everything after them. The rest of this path explains how it guesses so well, and where the guessing slips.

We said the model guesses the next “piece” of text. What exactly is a piece? It is called a token. A fixed procedure chops your text into tokens before the model sees anything: everyday words usually stay whole, while rarer or longer words are cut into familiar fragments, a bit like building a long word from small blocks.

Then each token is swapped for a number, its identification code in a fixed list. From that point on, the model sees only numbers, never letters. It does not look words up in a dictionary of meanings either; it receives a list of numbered chunks.

There is a subtlety about timing. When you send a long message, the model takes in all of its tokens together, in one batch. Only the reply comes out one token at a time. It is not a person reading your message left to right, one word at a time.

This is also why chatbots can fumble tasks that depend on the letters inside words: they never received the letters, only chunks. We take that puzzle apart properly in a separate series, Why can’t AI count the r’s in “strawberry”?

Who taught the machine to guess well? Nobody typed in grammar rules or facts. It learned by reading, an enormous amount of text from books, web pages and code.

The method is wonderfully simple. Take a real sentence, cover up the next token, and make the model guess it. The original text already contains the correct answer, so the grading is free: nobody has to label anything by hand.

Inside the model are billions of adjustable numbers called parameters, which you can picture as knobs on a giant mixing board. The best-known early large model, GPT-3, had 175 billion of them. At the start they are set at random and the guesses are gibberish. After each guess the model checks the real answer, and if it gave that token low odds, every knob gets a tiny push toward raising them. Each nudge is small, so that no single sentence overwrites what earlier text taught; the learning comes from an astronomical number of small pushes adding up.

Spelling gets learned first, then grammar, then facts and style, because all of it helps predict what comes next. When training ends, the knobs are frozen. What is left is a pattern of settings rather than a stored library of everything the model read, although a model can memorize some of its training text, and researchers have extracted verbatim passages from models.

Take the sentence: “The trophy didn’t fit in the suitcase because it was too big.” What does “it” refer to? To you it is obviously the trophy. To the model, the token “it” is just a number that is identical in countless sentences, so on its own it means almost nothing.

The mechanism that fixes this is called attention, and it was the central idea of the 2017 paper that introduced the transformer, the design behind today’s chatbots, built entirely on attention. Each token looks around and asks, in effect, which of the other tokens matter to me? It gives every other token a score, high for helpful ones and low for useless ones. For “it” in this sentence, “trophy”, “suitcase” and “big” get large weights, while little words like “the” get small ones. Training taught the model which clues usually point to the right noun.

Change the sentence to end “because it was too small” and the clue word changes, so the weight shifts toward the suitcase. The model is not simply linking “it” to the nearest noun; it is weighing the whole sentence.

Then “it” absorbs information from the words it weighted, in proportion, so the token now carries meaning from this particular sentence rather than a generic one. Every token does this at once, and the whole process is repeated through many layers. By the end, the last token has gathered the context it needs to guess what comes next.

Remember the training game: the model was scored on matching real text. Nobody scored it on whether a sentence was actually true. It was trained to be plausible, and correctness only comes along when true text happens to be the likely text.

Now think about what that means when you ask a question. The model does not open a file; it builds the answer one token at a time, choosing what fits. Ask it for a source and the most likely text looks like a citation, with an author, a journal and a year. The shape fits even if the paper does not exist.

There is also no built-in alarm for “I don’t know”. The model always produces a next token, and its tone stays equally confident whether the content is right or wrong. Where a fact appeared everywhere in the training text, likely and true usually match. Where it is rare, such as an exact number, a quote or a name, they drift apart.

This has had real consequences. In 2023 two New York lawyers were sanctioned after submitting a court filing that cited cases a chatbot had invented. Some chatbots can also search the web or use other tools, which helps, but the underlying habit remains. When a chatbot invents a detail, it is not broken or lying; it is doing exactly what it was trained to do. Treat fluent as a style, never as proof.

Every guess looks at the text so far, but there is a ceiling on how much text fits in view. That ceiling is called the context window, and it is measured in tokens. Its size varies between models and has grown over time.

Everything shares one budget: your messages, the model’s replies, any hidden instructions from the app, and even the answer it is writing right now. When the conversation outgrows the window, the oldest text slides out. The model is not ignoring it; for the next guess, it simply is not there.

Nor does the model store your chat. Each turn, the application sends the whole conversation back in, and the model reads it fresh. Its parameters are not changed by talking to you, so a brand-new chat starts with an empty window. (Some products add separate memory features that save notes and put them into the window, but that is the app doing it, not the model remembering.)

The practical advice follows. Put what matters close to where you need it. In a long chat that is drifting, start fresh and paste in a short summary of what counts.

After the first stage of training, the model is a pure continuer. Type “What is the capital of France?” and it might simply write more questions, because on the internet a question is often followed by more questions, like a quiz.

So developers add a second stage. They collect many examples of a request followed by a good answer, and train the model on them in the same way as before, by guessing the next token, but on carefully chosen examples of good exchanges.

A third stage uses human preferences. The model writes several answers to one prompt, people rank them from best to worst, and those rankings are used to build a scoring signal for what people prefer. The model’s numbers are then nudged toward replies that score well. OpenAI described this approach in its 2022 InstructGPT work.

Underneath, nothing has changed: it still guesses one token at a time. But now the most likely continuation of your request is a real attempt to help. Notice what is being rewarded, though: what people prefer. That usually goes along with helpful and accurate, but it is not the same as verified true.

Does the machine understand anything? Experts really do split. It handles new puzzles and can explain ideas in fresh ways, which looks like understanding. It also fails at some simple things and has no senses of its own, which looks like the opposite. The honest answer is that this is unresolved.

The useful point is that your practice does not depend on settling it. Whether or not it understands, it was trained to sound likely, not to check. Even if a future model clearly did understand, that would not guarantee that every detail it states was accurate or checked. So treat its output as a draft.

That does not make it useless. A cover letter, a summary or a trip plan in seconds is real value, and then you edit. A great draft is worth having. For tone, structure and brainstorming, the risk is low.

Check hardest on four things: facts, numbers, quotes and citations. A made-up statistic looks exactly like a real one, so if a chatbot says a study found some percentage, find the study and confirm the number before repeating it. Use it like a brilliant, fast, sometimes overconfident assistant: let it draft, and confirm what matters before it matters.