Verifying without expertise

The one study that measured this closely found the failure was not skipped checking: among participants who actively tried to verify, checking itself failed.

In a study published at CHI 2024, researchers hand-graded a chatbot's answers to 517 real programming questions and found 52% carried incorrect information.¹ Twelve programmers compared a chatbot answer with a human answer, question by question. They were not passive: all twelve pasted code into a local editor, most also used a sandbox, and ten said out loud that answers must be verified before use. They overlooked the wrong information anyway, 39.34% of the time; they preferred the chatbot answer about a third of the time, and 77.27% of the answers they preferred were incorrect.¹ They could tell which answer was machine-written 80.75% of the time, and that did not ensure they caught its errors: the 39.34% overlook rate held regardless.¹ Familiarity with the topic had no statistical relation to whether someone overlooked an error. The authors attribute the misses to answers that were comprehensive, polite and easy to read, which lowered people's guard.¹

That same paper names the condition under which checking broke down: verifying a claim by running code or reading long documentation left participants missing errors and underestimating how wrong the answer was.¹ The practical move is to sort a claim by what checking it costs, and the tree below lays out that sorting.

graph TD
  A["One answer"] --> B["Split into single claims"]
  B --> C{"What does this claim need?"}
  C -->|"a fact about the world"| D["Independent sources, new tab"]
  C -->|"code that must work"| E["Run it: demanding, misses here"]
  C -->|"long documentation"| F["Read it: demanding, misses here"]

Figure: One suggested way to sort a claim by the cost of checking it; no study has tested whether following this raises catch rates.

Leaving the text to check it is called lateral reading in the library-science literature. Professional fact-checkers verify an unfamiliar claim by checking what independent, established sources say about it, a habit found in fact-checkers but not in the historians or students used as comparisons.² A university library guide adapts this to chatbot output by splitting an answer into separate checkable claims, since there is no byline or single source to evaluate.³ A warning helps too: people given faulty assistance with no warning did worse on a knowledge test than people given none at all, and an explicit on-screen warning nearly doubled performance against the unwarned condition.⁴

Two limits on all of this. No study tests whether the advice above actually raises a non-expert's catch rate; the library sources describe the technique, and the human-factors studies measure failure without testing it. And experience is not the shield it sounds like. In a 2025 randomized trial of 44 licensed physicians using ChatGPT-4o recommendations, all having completed a twenty-hour course on evaluating AI output, diagnostic reasoning accuracy was 84.9% on unaltered recommendations and 73.3% when three of six cases carried deliberate errors, an adjusted drop of 14.0 percentage points; the half with at least ten years of practice lost more ground than the less experienced half, not less.⁵