Agent Task Decomposition: How Small Is Small Enough?

An agent given a large vague task wanders. The standard advice is to break the task into smaller ones, and it is good advice, but it is usually delivered without the part that matters: how small, split along which lines, and who does the splitting.

Decomposition is not free. Every boundary you draw costs a handoff, and every handoff loses context. Split too finely and you spend more tokens shuttling state between steps than you saved by making each step tractable.

Why decomposition helps at all

Three separate mechanisms are at work, and they have different implications.

The first is context economy. A subtask can be given a fraction of the working context — the three files it needs rather than the forty in the repository — which keeps the relevant material near the end of the prompt where attention is strongest. The lost-in-the-middle effect is a real cost of long undifferentiated prompts.

The second is verifiability. A small task can have a check. "Refactor the auth module" has no assertion; "make this test file pass without changing the test" does. Decomposition is often really a search for units that can be verified, and that reframing tells you where to cut.

The third is recoverability. If step four fails, you retry step four rather than the whole session. Without boundaries, a failure late in a long run means discarding everything or, worse, continuing from a corrupted state.

Cut along verification lines

The most reliable heuristic is to make each subtask end in a state you can check programmatically. If you cannot describe the check, the boundary is in the wrong place.

In practice this means a subtask usually corresponds to something like: a schema migration that applies cleanly, a module that compiles, a test that goes from red to green, an endpoint that returns the expected shape. These are natural units because the codebase already has machinery to verify them.

Subtasks that end in "understand the codebase" or "consider the options" have no check and tend to produce confident summaries that the next step then trusts. Research steps are legitimate, but treat their output as a hypothesis to verify rather than a fact to build on.

The overhead is real, so count it

Each boundary requires the parent to write a brief and the child to be given enough context to act. That brief is tokens, and the context handed down is usually a compressed and lossy version of what the parent knew.

A useful rule of thumb: if the context you must pass to a subtask is a large fraction of what the parent already had, the split is not buying you much. You have paid the handoff cost without gaining context economy.

Watch for the failure where a subtask is given too little and reconstructs it by reading files the parent already read. You now pay for the same file twice, and the two copies may disagree if anything changed. Agent memory and context management covers the plumbing for passing state without duplicating it.

Depth versus breadth

Two shapes of decomposition behave very differently. A flat list of six sibling subtasks is easy to reason about, easy to parallelise where dependencies allow, and easy to retry. A tree three levels deep is neither.

Deep hierarchies compound the lossy handoff. By the third level the leaf agent is working from a summary of a summary of the original intent, and the drift is invisible until the result comes back subtly wrong. Most systems are better off with one level of decomposition and larger leaves than with three levels and tiny ones.

Breadth has its own limit, which is the parent's ability to integrate results. Six subtask outputs arriving at once is a context problem for the parent, especially if each returns a wall of text. Require subtasks to return a short structured result rather than a transcript. Subagents and delegation covers the interface between levels.

Who decomposes, and when

There are three options and they are not equivalent.

A static decomposition, written by you as a workflow, is the most reliable. If your task type is known in advance — triage a bug report, then reproduce, then patch, then test — encode it. You get deterministic structure, per-step verification and easy observability, and you spend no tokens deciding the shape. This is the right answer far more often than it is used.

A model-generated plan up front suits tasks whose shape varies but is knowable from the request. The risk is that plans made before any exploration are made in ignorance, so they should be revisable rather than binding. ReAct versus plan-and-execute covers the trade-off.

Purely emergent decomposition, where the agent decides step by step, adapts best and is hardest to debug. It suits open-ended work with no known structure, and it is where turn ceilings and loop detection earn their keep.

Signs your split is wrong

Subtasks that consistently need to ask the parent for clarification are underspecified — the boundary was drawn where information was still entangled. Move it, or merge the two.

Subtasks that consistently exceed their turn budget were not actually decomposed; they inherited the original vagueness with a smaller title. "Fix the failing tests" is not smaller than "fix the bug" if there are forty failing tests.

And if two subtasks keep editing the same file, they were one subtask. Overlapping write scope is the clearest signal that a boundary is fictional, and it produces conflicts that are expensive to reconcile. Dependency-aware planning covers ordering the pieces once you have drawn them correctly.

A practical rule

Aim for subtasks that a competent developer could complete in one sitting without asking a question, that end in a check you can run, and that touch a disjoint set of files from their siblings. Prefer one level of decomposition with substantial leaves. Encode the structure statically wherever the task type repeats, and reserve model-generated plans for work whose shape genuinely varies.

When in doubt, split less than feels right and add boundaries only where you have watched a run fail for lack of one.