Agent Fan-Out Limits: How Wide Is Too Wide

Fan-out is the one multi-agent lever that feels free. Spawning twelve workers instead of four costs three times the tokens and roughly the same wall-clock time, so if quality scales with coverage, why not twenty?

Because quality stops scaling well before the cost does. There are four separate ceilings, they bind at different widths, and the one that bites first is almost never the one people plan for.

Ceiling one: the synthesis window

Every worker returns something the orchestrator must read. At four workers it reads carefully. At twenty it skims, and skimming means the coverage you paid for never reaches the answer.

This ceiling is set by the size of a result, not the number of workers. Twenty workers returning eight structured lines each is fine. Six workers returning two thousand words each is already past it. Fix the return schema before you touch the width, because tightening results is the cheapest way to raise this ceiling.

You can also reduce in a tree — merge in batches, then merge the merges — but each level compresses again, so buy width with a tree only when the work genuinely justifies it. The trade-offs are the same ones covered in map-reduce with agents.

Ceiling two: decomposition quality

A task has a natural number of independent parts. Comparing four candidate libraries has four. Asking one broad question has one.

When you fan out wider than the task decomposes, the orchestrator manufactures subtasks to fill the slots, and manufactured subtasks overlap. Three workers investigate the same file from slightly different angles and return three near-identical findings, which the orchestrator then has to spend judgement deduplicating.

This is why fan-out width should be an output of decomposition rather than a configuration value. Let the lead say how many independent branches exist and cap it, rather than telling it to produce eight.

The tell is easy to spot in logs: if worker results overlap heavily, you are over-fanned. If they contradict on facts, the briefs were ambiguous. Both are decomposition problems, not model problems.

Ceiling three: rate limits and concurrency

Twenty concurrent workers is twenty concurrent requests, each making multiple tool calls. That is not twenty requests, it is a few hundred over the life of the run, arriving in a burst.

What you get is 429s, then retries, then retries colliding with the workers that were still running. Wall-clock time goes up rather than down, and the token bill includes everything the failed attempts consumed. Rate limiting agent fleets covers the shared-budget side of this, and rate limits and retries the per-request mechanics.

The practical fix is a bounded worker pool rather than unbounded spawning: a fixed number of slots, workers queue for one, and the queue depth is visible. Wide fan-out with four slots is usually faster end to end than narrow fan-out with no limit, because nothing gets rejected.

Ceiling four: cost per unit of new information

The tenth worker rarely finds as much as the second. Coverage has diminishing returns and cost does not.

Anthropic reported that token usage alone explained about eighty percent of the performance variance in its research evaluations, and that its multi-agent system used roughly fifteen times the tokens of a chat interaction. Read the first figure carefully: spending more helps, which means a wide fan-out will usually look better than a narrow one in a quality comparison. The question is whether it looks fifteen times better.

Measure marginal value directly. Run the same task at three widths, and count findings that appear at the wider setting and not the narrower one. In most tasks the curve flattens sooner than the team expects, and the flattening point is your real limit.

Finding your number

Pick ten representative tasks. Run each at widths of one, three, six and twelve, with everything else held constant. For each run, record the token spend, the wall-clock time, the number of rejected requests, and a quality score from a rubric you wrote before you started.

Then plot quality against spend, not quality against width. Width is a knob; spend is the thing you are buying with. Almost always there is a point where the curve goes flat, and the width just below it is the setting to ship.

Re-run this when you change the model, the return schema or the tools. Any of those moves the ceiling, sometimes by a lot — a tool that returns ten relevant lines instead of a whole file can double the width the synthesis window supports.

Raising the ceiling instead of accepting it

Tighten the return schema. Fixed fields with hard length caps push far more workers through the synthesis step than free-form summaries do.

Deduplicate before synthesis. If results carry a location identifier, code can collapse duplicates and the orchestrator sees a shorter, cleaner set.

Give workers non-overlapping scopes explicitly, rather than trusting them to stay in their lane. A worker told which directories are out of scope will not rediscover its neighbour's findings.

Use a two-level tree only when a single synthesis genuinely cannot fit. It buys width at the cost of a second compression, and the second compression is where counts and identifiers get lost.

A default worth starting from

Start at three to five parallel workers, each with a hard step budget and a structured return of a few hundred tokens. That is the range Anthropic described for its research system, and it lands below all four ceilings for most tasks.

Increase only with evidence: a measured gain in findings, at a width your rate limits tolerate, with results your orchestrator demonstrably still reads. If the number of workers is a configuration value nobody has revisited since the prototype, it is almost certainly wrong in one direction or the other, and the cost of being wrong upward is the one that shows up on the bill. Agent cost control patterns has the rest of the levers.