How LLMs Write

AI that writes text builds a sentence by picking one next word at a time. Try that idea with a tiny table of our own. See the candidates for the next word and how likely each is, and find out how temperature (how adventurous the pick is) changes the result.

Temperature (higher = more surprising picks)

Theskyis

Candidates for the next word

blue 66%
wide 24%
high 10%

Press "Add one word".

Cut a sentence into tokens

Type a sentence

The 7
sky 8
is 9
blue 14
. 17
un 0
happi 1
ness 2

A small dictionary on this page, matched longest-first from the left (unknown characters one by one). The number is the dictionary order; dashed chips are characters not in the dictionary. Real tokenizers work differently.

A simplified teaching model. The table is homemade and is not a real language model.

How to play

Your goal is to watch a sentence grow one word at a time and see how temperature changes the result. First choose an opening and a temperature. Press "Add one word" and the candidates for the next word appear as bars showing how likely each is. One is drawn by lot and added to the sentence. Repeat to build a sentence. The higher the temperature, the more often unlikely words get picked. "Auto" builds the same opening three times, at temperature 0, 1 and 2. At the bottom you can cut your own sentence into tokens (small pieces of characters or words).

Learn more

A text-generating program (a language model) takes the words so far, produces a probability for each possible next word (token), draws one, adds it to the text, and repeats. A token is a unit the program works with: a character, a word, or a piece smaller than a word. The probability table on this screen is a tiny one we made ourselves. It is not output from a real language model, nor the result of a real tokenizer. Real models learn tendencies from large amounts of text: which words tend to follow which. They do not look up a table; a neural network computes the probabilities, using far longer context than here. This screen shows only the part where one is drawn from the probabilities. The probabilities come from dividing the scores by a temperature T and applying softmax: p = exp(z/T) / sum of exp(z/T). For example, scores [2.0, 1.0, 0.1] give about [0.659, 0.242, 0.099] at T=1, about [0.864, 0.117, 0.019] at T=0.5, and about [0.502, 0.304, 0.194] at T=2. A smaller T favours the largest more; a larger T flattens the odds. T=0 would divide by zero in the formula, so it is treated separately as "pick the largest". The random seed is fixed, so the same actions give the same results. What matters is that a model is built to pick a plausible next word, and has no mechanism of its own for checking whether each sentence is true. So it can write something plausible that does not match the facts (researchers call this "hallucination"). "Poko Town" is made up, but the table does not check whether the town exists; it simply continues with apples or oranges. There are ways to combine a model with outside information to check, but that is separate from how a model works on its own. That does not mean it is always wrong, nor that it is always right: because it learned tendencies, it tends to do well on common topics and to invent on things that cannot be checked. The way text is cut into tokens in real systems is different from this screen's small dictionary. A simplified teaching model.