目录 The size of one task

Working a task

The size of one task

The right size for one agent task has not been measured. What exists is one benchmark curve about task duration, plus vendor scoping advice, and neither of them tells you how large a change your repository can absorb in a single session.

METR measured a time horizon for frontier agents: the task length, in human time, that they complete with 50 percent reliability, which has roughly doubled every seven months over six years.1 In the same work success falls steeply with length: near 100 percent on tasks a human finishes in under four minutes, and under 10 percent past roughly four hours, measured in March 2025 against a Claude 3.7 Sonnet era model whose 50 percent point sat near one hour.1 On an alternative dataset the doubling was faster than three months.1 These are benchmark suites with human baselines. No source maps that curve onto work in your own repository, so read it as a direction, not as a budget.

Vendor advice fills the space where measurement is absent. GitHub names the work its coding agent suits, including bug fixes, test coverage, documentation and accessibility, and names what to keep off it, including broad refactors needing cross-repository knowledge, production-critical incidents, security and PII work, and anything with ambiguous requirements.2 It defines a well-scoped issue as three things: a clear problem description, complete acceptance criteria, and directions about which files change.2 OpenAI decomposes long-horizon work into "milestones small enough to complete in one loop", each with acceptance criteria and validation commands.3 DORA reports that working in small batches amplifies the benefit of AI adoption and shortens feedback time on changes, but that is correlational survey data about teams, not an experiment varying agent task size.4 Keep tasks small and atomic is widely recommended and not measured.

Claude Code's stated reason is about context rather than task count: the window fills fast and performance degrades as it fills, and it names the unscoped investigation that reads hundreds of files as its own failure pattern.5

No source establishes an optimal slice size, which is where this judgement stops: smaller is not proven universally better. METR's randomized controlled trial (16 experienced open-source developers completing 246 tasks) found the same developers 19 percent slower to finish with AI tooling allowed, while they believed themselves faster; it tested early-2025 chat and IDE assistance rather than autonomous agents.6 It argues for measuring your own case, not for any particular slice size.

参考文献

测验
  1. What do METR's time-horizon numbers actually measure?

    • Benchmark tasks with human baselines
    • The largest diff an agent can write before its reliability drops in a production repository
    • The number of files a coding agent may safely touch in one session

    METR timed agents against benchmark suites with human baselines. Nothing in that work maps the curve onto your own repository, so it gives a direction rather than a diff-size or file-count limit.

  2. A repository task should be sized using a vendor-published ceiling on lines changed or files touched.

    • True
    • False

    No vendor publishes such a ceiling, so sizing rests on scoping judgement instead: a clear problem description, acceptance criteria and named files, not a numeric threshold.

  3. DORA measured small batches through surveys and correlations across teams. Applied to sizing one agent task, that finding functions as ____ evidence.

    • directional
    • load-bearing
    • conclusive

    DORA's result is survey data about team performance, not an experiment varying agent task size, so it points a direction rather than setting a threshold. This research found no agent-specific experiment on task size.

评论

还没有评论,来说第一句吧。