What changes and what will not

Three different things get called limits of these systems, and only one of them is documented as coming from the architecture.

The architectural one is the cost of length. The 2017 paper that introduced the design records that the work each layer does to relate every position in the text to every other grows with the square of the length, against growing in proportion to the length for the designs it replaced.¹ That is why a long conversation costs rather than being free. The same authors, in the same section, proposed restricting each position's attention to a neighbourhood around it, to improve performance on very long sequences.¹ The cost was an engineering target for the people who first described it.

The size of a window is a training choice. A 2023 paper took open models already trained with a 2048-token window and ran them at up to 32768 tokens by rescaling position indices, fine-tuning inside a thousand steps, with the same weights.² Today's figures are dated product facts, true as of 2026-09-12: a one-million-token window on current flagship Claude models, 200,000 on others, and a plain error when the input exceeds it.³

Statelessness is a default with layers built on top. Each request is documented as isolated, "each text generation request is independent and stateless", with multi-turn conversation implemented by resending the history.⁴ On top of that, OpenAI documents a conversations object holding state across sessions and devices, and Anthropic documents context editing, which clears older tool results past a threshold, plus a memory tool that writes to files outside the window and reads them back.⁴⁵ The per-request window is unchanged; what is kept inside it changes.

One widely repeated claim is contradicted by a vendor's own research: that fabrication is a permanent, unfixable limit. A September 2025 OpenAI paper instead locates it in ordinary statistics and in evaluation that scores accuracy alone, which rewards a confident guess over an admission of uncertainty, and argues that the incentive rather than the architecture is what would have to change.⁶

Here is where this page stops. No source found states whether any of these limits is permanent in principle: nobody says the cost of length cannot be avoided by another design, and nobody says it can. The record does show that limits loudest a few years ago were moved, a reason to be careful with the word permanent rather than a claim about what comes next. Anthropic also documents that accuracy and recall degrade as the token count grows inside a window, an effect it calls context rot; no source found explains its cause, so this page reports the effect and leaves it unexplained.³