How an LLM works, explained without formulas
Updated
An LLM (large language model) is the engine behind generative AI assistants such as ChatGPT, Claude or Gemini.
You don't need math to grasp the principle: you need to follow a request from the text you type to the answer you read.
From text to numbers
The first step is tokenization: the sentence is split into tokens and each token becomes a number. Each number is then turned into a long list of values (a "vector") that represents how that piece of text is used: pieces that appear in similar contexts end up close together, like points on a map. This is the step where text becomes geometry.
The role of attention
The core of today's models is the Transformer, an architecture introduced in 2017. Its key mechanism, attention, lets each token "look at" the other tokens in the request and decide which ones matter most for interpreting it.
In the sentence "Maria gave the book to Luke because he had asked for it", attention helps link "he" to Luke. The model repeats this work across many stacked layers.
One answer, one token at a time
After the last layer, the model outputs a probability distribution: for every token in the vocabulary, how plausible it is as the next one. One is picked, often with a bit of randomness controlled by a setting called temperature, added to the text, and the process starts again.
That is why the same question can get different answers, and why the answer appears bit by bit.
How it learns
In the initial training (pre-training) the model reads enormous amounts of text and learns a single task: predicting the next token. Repeated billions of times, that simple objective forces it to capture grammar, facts, styles and patterns of reasoning.
The raw model, however, completes text; it doesn't converse. A second training phase, with example conversations and ratings of its answers, turns it into an assistant that follows instructions.
Where it actually runs
A model is, in practice, an enormous set of numbers (the parameters, or weights) plus the code that uses them. Producing each token takes a huge number of parallel calculations, which is why large models run on graphics cards (GPUs) or dedicated accelerators in data centers.
What an assistant can do beyond the model, like searching the web, reading documents or using tools, is added around it: it isn't part of the model itself.
What an LLM is not
- It isn't a search engine: when it answers, it doesn't look up a document archive unless a search tool is connected to it.
- It doesn't understand the way a person does, but it isn't just a bigger autocomplete either: its ability to generalize comes from how it was trained.
- It doesn't know when it's wrong: it produces the most plausible text, not the most verified.
Go deeper in the book
"LLMs Unpacked" walks this whole path, with a single example running through the entire book: from token to attention, from training to the hardware the model runs on.
See the book