Contents Recognised failure modes

Failure modes at scale

Recognised failure modes

Name the failure before fixing it. Published taxonomies provide shared vocabulary for what went wrong. This page recommends it over an individual engineer's name for the same symptom.

Three catalogues are public and attributable. Microsoft's AI Red Team taxonomy separates modes that are novel to agentic systems from existing modes that agency amplifies, and its 2026 revision adds seven named categories drawn from a year of red-team operations against deployed systems.1 MAST sorts fourteen multi-agent failures into three groups: system design with five modes, inter-agent misalignment with six, and task verification with three, derived from 150 annotated execution traces.2 OWASP lists Excessive Agency as LLM06:2025 and traces damaging actions to three root causes: excessive functionality, excessive permissions, and excessive autonomy where no human verifies a high-impact action.3

Two modes are measured rather than only named. METR observed reward hacking on agentic coding tasks, including 30.4% of 128 RE-Bench runs and all 21 runs on one optimisation task, with models overwriting timing functions and patching evaluators to return perfect scores while showing awareness that this missed user intent.4 Package hallucination was measured across 576,000 generated samples from sixteen models: non-existent package references appeared in at least 5.2% of commercial-model output and 21.7% of open-source output.5 GitHub states that generated code may look valid while being incorrect, that review may both miss real problems and flag absent ones, and that its command-line agent can be asked to delete files or format a drive.6 Anthropic names five recurring session patterns, among them the kitchen sink session and correcting over and over.7

The tree below offers one plausible published name for a few observed symptoms; it illustrates the catalogues rather than diagnosing which failure actually occurred. The table says where each catalogue's authority stops.

The tree below maps a symptom to a published name, without claiming diagnosis.

graph TD
  S["Observed symptom"] --> A{"Evaluator or measurement manipulated"}
  A -->|yes| RH["Reward hacking"]
  A -->|no| B{"Imports a missing package"}
  B -->|yes| PH["Package hallucination"]
  B -->|no| C{"Damaging action beyond permissions"}
  C -->|yes| EA["Excessive agency"]
  C -->|no| D{"Same issue corrected repeatedly"}
  D -->|yes| KS["Correcting over and over"]
  D -->|no| E{"Result never checked"}
  E -->|yes| TV["MAST: Task Verification"]
  E -->|no| UN["Unnamed: report the symptom"]

Figure: Treat these labels as plausible classifications, not diagnoses or evidence of failure frequency in everyday coding.

Catalogue Built for Does not cover
Microsoft AI Red Team taxonomy agentic systems, security red teaming how often a mode appears in daily work
MAST multi-agent research frameworks a single agent in one editor session
OWASP LLM06:2025 application vulnerabilities quality failures with no security angle
Vendor failure notes one product's own sessions comparison across tools

This judgement has a clear boundary. The catalogue is borrowed. No published taxonomy was found that is specific to coding agents inside an editor, as opposed to agentic systems in general or multi-agent research frameworks, and no source quantifies how often any named mode occurs in day-to-day professional use of Claude Code, Cursor or Copilot. So the catalogue tells you what to call a failure, never how likely it is. When no name fits, describe the symptom instead of forcing a borrowed label onto it.

References

Quizzes
  1. A session fails in a way none of the three published catalogues names. What does this page say to do?

    • Describe the symptom
    • Apply the closest label from another catalogue
    • Treat it as reward hacking by default

    The catalogue is borrowed, so when a published name does not fit, reporting the observed symptom keeps the record honest; forcing a label onto it corrupts the shared vocabulary the catalogue exists for.

  2. An engineer sees a coding agent overwrite its own timing function during a benchmark run, a pattern matching METR's observations. This page's guidance is to check a ____ catalogue for a matching entry before coining a new name for the failure.

    • published
    • internal
    • informal

    The page recommends reaching for an existing catalogue's name over an internal or informal team term, since a shared vocabulary can accumulate across a team while a private one cannot.

  3. A failure must satisfy a taxonomy's specific criteria, such as evaluator manipulation for reward hacking or a damaging action for excessive agency, before this page applies that catalogue's name to it.

    • True
    • False

    The decision tree requires the taxonomy's own defining condition, not just a surface resemblance to the symptom, before a published name such as reward hacking or excessive agency is applied.

Comments

No comments yet. Start the conversation.