How ChatGPT Works
Type a question into ChatGPT and a fluent answer appears, one word at a time. It feels like the system understands you. Underneath, a large language model does something much simpler to state: it reads the text so far and predicts what piece of text should come next, then repeats that over and over.
How do large language models like ChatGPT work? They break your text into small pieces called tokens, turn each token into a list of numbers, pass those numbers through many layers of a neural network called a transformer, and use the result to score every possible next token. One is picked, added to the text, and the whole loop runs again.
Step 1: Text becomes tokens
A model can't read letters the way you do. First, a tokenizer splits your text into chunks drawn from a fixed vocabulary. A common word may be a single token, while a rare word is split into several fragments. Each token maps to an ID number.
That's why these models sometimes stumble on tasks like counting the letters in a word: they never see the word as letters, only as a few chunks.
Step 2: Tokens become vectors
Each token ID is looked up in a table and replaced by a long list of numbers called an embedding. These numbers aren't hand-written. They are learned during training, and tokens that get used in similar ways tend to end up with similar embeddings. Position information is added too, so the model knows the order of the words.
Step 3: The transformer mixes context
The core of a large language model is a stack of transformer layers. Each layer does two main things:
- Attention lets every token look at the tokens before it and decide which ones matter for its meaning. In "The trophy didn't fit in the suitcase because it was too big," attention is how "it" can lean on "trophy" rather than "suitcase".
- A feed-forward network then transforms each token's vector further, layer by layer.
After many layers, each token's vector has absorbed a lot of context from the words around it.
Step 4: Predicting the next token
At the end of the stack, the model turns the final vector into a score for every token in its vocabulary, which becomes a probability distribution. "The capital of France is" puts very high probability on " Paris" and tiny probability on most everything else.
The system then picks a token. Always taking the top choice gives predictable text; sampling from the distribution, with a setting often called temperature, gives more varied text. The chosen token is appended and the loop repeats until the answer ends. This is why you see text stream in word by word.
How a model learns to do this
All the numbers inside the network, often billions of them, are called parameters. They start out essentially random. During pre-training, the model is shown huge amounts of text and asked to predict each next token. Whenever it's wrong, a procedure called gradient descent nudges the parameters slightly so that the right answer becomes a bit more likely. Repeat that an enormous number of times and the network ends up capturing grammar, facts, and patterns of reasoning, because all of those help predict text.
A raw pre-trained model is a text continuer, not an assistant. Chat products then add further training on examples of helpful conversations and on human feedback about which answers are better, which steers the model toward following instructions. Exact recipes vary by company and aren't something to guess at.
What this explains, and what it doesn't
Knowing the loop explains a few behaviours:
- Confident mistakes. The model is optimised to produce plausible text, not to check facts, so it can state something false in a fluent voice.
- A context window. The model can only attend to a limited amount of text at once, so very long conversations eventually push early material out.
- No lookup by default. Knowledge lives in the parameters learned during training, unless the product adds a search or document tool.
Some newer models route each token through only part of the network; our post on mixture-of-experts routing covers that variation.
Quick recap
- A large language model predicts the next token, then repeats.
- Text is split into tokens and turned into vectors of numbers.
- Transformer layers use attention to mix in context from earlier tokens.
- Training adjusts billions of parameters to make real next tokens more likely.
- Chat behaviour comes from extra training on top of the raw predictor.