Agent Token Budgets: Capping Spend Without Capping Capability

An agent given a task and a set of tools has no inherent notion of what it should cost. It will keep working as long as the loop lets it, and a single confused session can consume more than a hundred successful ones.

Budgets are the control. Setting them badly is common, and the two failure modes are opposite: too tight and legitimate work fails, too loose and nothing is actually bounded.

Why turn limits are a poor proxy

The usual first control is a maximum turn count, and it is weak because turns are not uniform.

A turn that reads a small file costs almost nothing. A turn that dumps 50,000 tokens of test output costs a great deal. Twenty turns can mean anything across two orders of magnitude, which makes a turn limit an unreliable cap on spend.

Turn limits do have a role as a runaway guard, because they stop infinite loops. They are a safety net, not a budget.

Budget in tokens, and count everything

Track cumulative tokens across the session — input and output, every turn — and stop when the total exceeds a ceiling.

The subtlety people miss is that input dominates. Because the whole transcript is resent every turn, cumulative input grows quadratically with session length: turn twenty resends everything from turns one to nineteen. A session with modest output can still be expensive.

Count what you are actually billed for, which means separating cached from uncached input if your provider prices them differently. A long cached prefix costs a fraction of the same tokens uncached, and a budget that ignores this will stop sessions unnecessarily. Prompt caching savings math covers the arithmetic.

Set the ceiling from data

Do not guess. Run a representative sample of real tasks with a very high limit and record the distribution of tokens consumed by successful runs.

Set the ceiling above the 95th percentile of successes. That stops the pathological tail while leaving legitimate hard tasks room to finish.

The distribution is usually heavily skewed — most tasks finish well under the median, a few take many times longer. A ceiling set at the mean would kill a meaningful share of work that would have succeeded, which is why percentiles rather than averages are the right basis.

Warn before stopping

A hard stop at the limit wastes everything spent so far. A warning first often converts a doomed session into a finished one.

At around 80 percent of budget, inject a message telling the model it is approaching its limit and should prioritise completing or summarising its progress. Models respond to this sensibly: they stop exploring and consolidate.

Even when the task cannot be finished, a summary of what was learned makes the next attempt much cheaper — it starts from established facts rather than from nothing.

Different budgets for different work

A single global ceiling is either too tight for hard tasks or too loose for easy ones.

Classify tasks on entry, cheaply, and assign a budget accordingly. A one-line fix and a cross-cutting refactor should not share a limit. The classification does not need to be sophisticated — task type, file count, or an explicit caller-supplied tier all work.

Where a task genuinely needs more, an escalation path is better than a higher default: the agent requests more budget, and either a policy or a human grants it. That keeps the common case bounded while allowing the exception.

Budget as a signal, not just a limit

The most useful thing about token budgets is what exceeding them tells you.

A task that reliably exhausts its budget is usually not a budget problem. It is a task the agent cannot do with its current tools, a prompt that fails to constrain it, or a tool returning far too much output. Raising the ceiling hides that.

Log budget exhaustions with the task type and inspect them weekly. They cluster, and the cluster points at a fixable cause far more often than at genuinely hard work. Agent cost control patterns covers the broader set of levers.

Budgets across a fleet, not just a session

A per-session ceiling bounds one agent. It does nothing about a hundred agents each staying comfortably inside their limit while collectively spending far more than intended.

Fleet-level control needs a second layer: a rolling spend total across all sessions in a window, with a policy for what happens when it is approached. Options are queueing new work rather than starting it, downgrading everything to a cheaper model tier, or refusing non-urgent tasks until the window rolls over.

The distinction matters because the two limits fail differently. A session ceiling protects you from one pathological task. A fleet ceiling protects you from a deployment that accidentally triples the number of concurrent agents, which is a far more common way to be surprised by a bill.

Instrument both, and alert on the fleet total rather than on individual sessions. Individual sessions exceeding budget are routine and mostly uninteresting; the aggregate crossing a threshold is the thing that needs a human. Alerting on token spikes covers what to watch.