Eigma
EN 中 DE
What Is Inside a Chatbot Open Eigma
Contents Context is the lever

Getting better answers

Context is the lever

One change a reader can make is to let a short question carry the situation it came from. Anthropic recommends explaining the context or motivation behind an instruction, saying it helps the model understand the goal and deliver a more targeted response1. OpenAI recommends a related but different practice, supplying reference text, useful when the answer depends on material the model never trained on2. Neither is a claim that this beats other changes; it is what each vendor happens to advise.

What has been measured is narrower than that, and it is worth knowing which part. In a 2023 controlled experiment on multi-document question answering, the same twenty documents were reordered around the one holding the answer. With that document first, GPT-3.5-Turbo scored 75.8 percent; ninth, in the middle, 53.8 percent; last, 63.2 percent3. Asking the question with no documents at all scored 56.1 percent, so for GPT-3.5-Turbo in that setup a buried answer left the model worse off than supplying nothing3. Length erodes it further: a 2025 evaluation of twelve models found ten of them keeping only half their short-context accuracy at thirty-two thousand tokens on a task that required a non-literal match4. The largest effect found anywhere in this research comes from retrieval rather than wording: on FreshQA, a 2023 benchmark of questions whose answers change over time, GPT-4 answered 28.6 percent correctly under strict grading with no retrieved context and 75.6 percent when search results were placed in the prompt5.

Notice what none of that covers. Those studies vary the position of a passage, the number of documents, the length of the prompt, and retrieved text against none. Adding two or three sentences of your own circumstances to a short everyday question is what this page is about, and no source found measures it. The advice is vendor-recommended, and for that everyday case it remains unmeasured.

The boundary is that context is not free. In a 2024 retrieval experiment on four small open models, Llama2-7B, MPT-7B, Phi-2 and Falcon-7B, adding a single document that was topically related but did not contain the answer cut accuracy by roughly a quarter, while wholly unrelated text sometimes helped instead6. In that setup, plausible but wrong material cost more than material with nothing to do with the question.

References

  1. [1]Anthropic prompting best practices
  2. [2]OpenAI prompt engineering guide
  3. [3]Lost in the Middle, Liu et al. 2023
  4. [4]NoLiMa, 2025
  5. [5]FreshLLMs and FreshQA
  6. [6]The Power of Noise
Quizzes

  1. What risk does the evidence point to when the passage containing the answer is buried among many other documents in a prompt?

    • The answer can end up harder to find than if no documents had been supplied at all
    • The model always ignores every document placed after the question
    • The model automatically shortens documents placed in the middle

    With the answer document ninth of twenty, GPT-3.5-Turbo scored below its own closed-book baseline. Burying relevant material can cost more than leaving it out.

  2. A document that is topically related to a question but does not contain the answer is safer to include than a document that is completely unrelated to it.

    • True
    • False

    A retrieval experiment on small open models found the opposite: a topically related document lacking the answer cut accuracy by roughly a quarter, while a wholly unrelated one sometimes helped.

  3. With twenty documents and the answer buried in the middle, GPT-3.5-Turbo's accuracy landed ____ its own closed-book score for asking with no documents.

    • below
    • above
    • exactly at

    Fifty-three point eight percent against a closed-book fifty-six point one, for GPT-3.5-Turbo in that twenty-document setup where the answer sat in the middle.

Comments

No comments yet. Start the conversation.