Stop Calling LLMs Next-Token Predictors
Stop Thinking of LLMs as Next-Token Predictors
The common description of large language models as 'next-token predictors' is technically accurate but misleading. While LLMs do emit tokens one by one, modern post-training techniques like reinforcement learning with verifiable rewards (RLVR) enable them to learn from novel sequences generated through exploration, not just from existing text. Using a chess analogy, the author distinguishes between a system that predicts the next move in a database of grandmaster games and one that chooses moves to maximize winning probability. This shift in learning objective means LLMs can encode knowledge and behaviors never seen in their training data, making the next-token framing an incomplete mental model.
A simulation of a helpful assistant and knowledge discovered through exploration can both be encoded in exactly the same next-token loop.
- mofeien
Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step is doubly wrong:
1. In order to select even the first word of a meaningful sentence, it already has to have structure and meaning of what follows captured somewhere inside, mostly in it's weights/activations or indexed by it's state vector.
2. What you see when you use an LLM is not next-token prediction directly next to the prompt, but instead following a block of varying length of next-token prediction that happened to make progress on the problem in your prompt, and which just summarizes the results.
- Systemerror7A69
To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well.
The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into it - but at least I would still call it a "next token predictor"
It's not using just training data, but what it's doing is predicting the next token to get to the solution. As far as my amateur knowledge goes, LLMs still roughly go token by token, deciding which one fits best given the context.
It's just not predicting based on it's training data, but predicting based on RLVR & more, trying to get to the optimal solution ( as much as the solutions CAN be optimal)
And I honestly think keeping this very much in mind is helpful in understanding and dealing with LLMs.
- bluegatty
'next token predictor' is a limited mental model but it's actually much better than any others.
'pattern matching' is a better intuition that 'reasoning' even though I think nominally, using the term 'reasoning' is perfectly fine in that context. It's just a loaded word that brings too much to the table.
'It hasn't seen the pattern' is a better description of the limitations of AI because it really just doesn't generalize very well at all. The adaptations described in the article don't change that.
Those are mutations, not expansions of capability.
- Planktonne
I'm not going to stop describing things accurately because someone generated an article that continually undermines its own main point. Limiting the way we talk and think about LLMs to a very narrow set of terms doesn't help us.
EDIT: gentler phrasing
- a3w
The errors LLMs typically made for me were looking for "schmutz" as a jiddish word, got "schmuck" on my screen. Thought why the stupid mistake? The next-token predictor model perfectly explains it.
Or starting with "yes". And this early locking in was a total lie, in the discussion that became "yes, might appear that way, but totally no since reasons". So it should have written "No", topmost, but could not self-edit that.
But nice that this gives it a more nuanced view, I might have to update my priors.
- stanleykm
> Calling the second system a “next-move predictor” would be strange. It is not trying to predict what move appeared next in a dataset. It is trying to choose a move that wins.
i dont understand the distinction here. does working backwards from a set of win states instead of working forwards from the current state somehow change whether it’s a prediction or not?
- teekert
It’s written in Claudish, or perhaps a human who has been reading too much Claudish recently. I am starting to become allergic to Claudish. Not there fully yet — but it’s at a tipping point genuinely worth exploring and worth being precise about.
- jvanderbot
A better statement might be:
Current agentic systems may be *built* from next-token predictors which are conceptually simple, but because of agentic frameworks, recursive invocation, tool use, and *heavy* investment in reinforcement learning in these contexts and for specific applications, they can no longer be thought of as "Merely" next token predictors.
Modern agentic work is probably more of a "emergent system from simple rules and complex interactions" paradigm than a genuinely new technology.