Contents Checkable definitions of done

Why agents fail

Checkable definitions of done

Anthropic documents the mechanism directly: Claude stops when the work looks done. If nothing in the session can tell looking done from being done, that impression is the only signal available, and the person reading the result becomes the verification loop, so every mistake waits for a human to notice it.1 The fix is to state completion as something the agent can run.

What counts as a check is narrower than it sounds and wider than a test suite. Anything that returns a signal the agent can read qualifies: a test run, a build exit code, a linter, a script that diffs output against a fixture, a screenshot compared against the design it was meant to match.1 Anthropic's documented example shows how small such a rewrite can be: a request to validate email addresses becomes a named function, three concrete cases with their expected answers, and an instruction to run the tests afterwards.1 For interface work the checkable form is the screenshot loop: implement, capture, compare against the original, list the differences, fix them.1 GitHub's custom instructions carry the same role, down to recording which commands do not work and why.2

How hard the criterion binds is a choice with documented steps. You can ask for the check in the prompt; set it as a goal condition that a separate evaluator re-checks each turn; run it from a hook that blocks the turn from ending until it passes; or hand grading to a reviewer that did not do the work.1 Hooks are the deterministic tier because instructions in a brief are advisory while a hook always runs.1 The gate is not absolute: Claude Code overrides a blocking stop hook after eight consecutive blocks, and a stalled goal eventually ends the run with the goal still set.1

Here is the hole, and it is not small. No vendor defines a checkable criterion for non-functional work. A refactor, a rename, a documentation pass or an architectural change has no test that separates done from not-done, and no source closes that gap. No vendor documents a checkable completion criterion for that work, and human review sits outside the runnable-check approach this page describes: the thesis of this page does not reach it. There is also no published measurement of what adding a check is worth; the guidance is documented practice, including the blunt version, "If you can't verify it, don't ship it."1

References

Quizzes
  1. A definition of done is usable when it is something the agent can ____ inside the session.

    • run
    • restate
    • estimate

    Anything returning a signal the agent reads qualifies: a test, a build exit code, a linter, a fixture diff, a screenshot compared against the design. No vendor documents such a signal for refactoring work.

  2. Naming a hook that blocks the turn is enough to make a criterion checkable, even if nothing in the session returns a pass or fail signal.

    • True
    • False

    A checkable criterion needs a signal the agent can read, such as a test or build exit code; a hook only enforces that the check runs, and Claude Code itself overrides a blocking stop hook after eight consecutive blocks.

Comments

No comments yet. Start the conversation.