What the thing is
Next-word prediction
The step underneath everything a chatbot does is one prediction: given the text so far, what comes next. OpenAI's technical report for GPT-4 says the model was pre-trained to predict the next token in a document,1 and Hugging Face's course states the same task without jargon, predicting the next word in a sentence having read the previous ones.2 Nothing about a question or an answer is built into this. The model was trained on raw text without labels and came out of it with a statistical grasp of the language it read.2
The prediction only ever looks backwards. This is called causal language modelling, because the output depends on past and present inputs but not on future ones,2 and in this family of models the attention layers for a given word can reach only the words before it.3 Other objectives exist. A model can instead be trained to fill in a word hidden in the middle of a sentence.2 Chat models use the backward-looking kind.
What the model emits is not a word. The architecture paper describes the final stage as converting the model's output into predicted next-token probabilities,4 one for every token it could produce. A separate step then picks one of them. Framework documentation calls the default choice greedy, taking the most likely token, and exposes sampling and a temperature setting that make the pick less predictable.5 That step is a knob rather than part of the model, and it is why one question can come back worded differently twice. The documentation says so directly: generation involves randomness, so results need not repeat.6 The chosen token is then fed back in and the same prediction runs again over the model's own output, until a stopping token or a length limit.5 Anthropic puts the visible behaviour in one line: "Claude writes text one word at a time."7
The boundary. Emitting one token at a time is not the same as thinking one token ahead, and the popular version of this page's claim gets that wrong. Anthropic's interpretability work finds a model planning many words ahead and writing in order to reach that destination, and in a rhyming task it settles on candidate rhyming words before starting the line they have to end.7 The loop above is also how these models work in general, taken from framework documentation. No vendor publishes the selection settings behind a shipped product, so none of it is a named product's configuration. The figure below sketches that scored-probability step as schematic bars over a handful of candidate next tokens for one unfinished sentence, with bar lengths standing for relative likelihood only and matching no published numbers.
<svg viewBox='0 0 700 250' width='700' height='250' xmlns='http://www.w3.org/2000/svg' font-family='system-ui, sans-serif' fill='currentColor'>
<title>Schematic probability bars over five candidate next tokens for one unfinished sentence</title>
<text x='12' y='22' font-size='14'>Unfinished: she poured the tea into the ...</text>
<text x='140' y='64' font-size='14' text-anchor='end'>cup</text>
<rect x='150' y='48' width='300' height='22' rx='4' fill='currentColor' fill-opacity='0.35' stroke='currentColor' stroke-opacity='0.55'/>
<text x='140' y='98' font-size='14' text-anchor='end'>mug</text>
<rect x='150' y='82' width='210' height='22' rx='4' fill='currentColor' fill-opacity='0.35' stroke='currentColor' stroke-opacity='0.55'/>
<text x='140' y='132' font-size='14' text-anchor='end'>pot</text>
<rect x='150' y='116' width='138' height='22' rx='4' fill='currentColor' fill-opacity='0.35' stroke='currentColor' stroke-opacity='0.55'/>
<text x='140' y='166' font-size='14' text-anchor='end'>sink</text>
<rect x='150' y='150' width='74' height='22' rx='4' fill='currentColor' fill-opacity='0.35' stroke='currentColor' stroke-opacity='0.55'/>
<text x='140' y='200' font-size='14' text-anchor='end'>every other token</text>
<rect x='150' y='184' width='44' height='22' rx='4' fill='currentColor' fill-opacity='0.35' stroke='currentColor' stroke-opacity='0.55'/>
<text x='12' y='228' font-size='13'>The model scores every token it can emit, not only the four named here.</text>
<text x='12' y='244' font-size='13'>Illustrative. Bar lengths are shapes, not values anyone published.</text>
</svg>
Figure: Schematic probability bars for one unfinished sentence; lengths are illustrative, not published values.
参考文献
- [1]OpenAI, GPT-4 technical report
- [2]Hugging Face LLM course: what language models do
- [3]Hugging Face LLM course: decoder models
- [4]Vaswani and others, the transformer architecture
- [5]Hugging Face Transformers: text generation
- [6]Hugging Face LLM course: generation varies
- [7]Anthropic: tracing the thoughts of a language model
测验
At each step of writing an answer, what does the model itself produce?
- A probability for every token it can emit
- The single word it has settled on
- A sentence it then trims to fit
The final stage turns the model's output into next-token probabilities across everything it could emit. Choosing one of them is a later and separate step.
Framework documentation exposes greedy selection versus sampling and a temperature setting, showing that picking a token from the probabilities is a ____ choice rather than a fixed part of the model.
- configurable
- encoded in the model's weights
- identical for every request
Sampling and temperature are settings a developer sets separately from the trained model, which is why identical input can come back worded differently on different requests.
评论
还没有评论，来说第一句吧。