Cost Per Agent Run: Why Input Dominates the Bill

Teams estimating agent cost usually reason about output. The agent writes some code, that is a few thousand tokens, at output rates that is a manageable number. Then the invoice arrives and it is several times larger than expected.

The reason is structural, and once you see it the arithmetic stops being surprising.

The transcript is resent every turn

A model is stateless between calls. Everything the agent knows on turn twenty is in the message list sent on turn twenty, which means the entire accumulated history goes over the wire again on every single step.

So cumulative input does not grow linearly with turns. It grows roughly quadratically. Turn one sends the system prompt. Turn twenty sends the system prompt plus nineteen turns of accumulated tool output and reasoning.

A thirty-turn session with an average transcript of 40,000 tokens sends about 1.2 million input tokens in total, while perhaps generating 15,000 output tokens. Even at output rates several times higher than input rates, input dominates by a wide margin.

Agent loop anatomy covers why the loop is built this way.

Working an example

Take a coding agent on Kimi K3, which lists at $3 per million input tokens and $15 per million output, with cached input at $0.30.

Assume a 12,000-token system prompt with tool definitions, twenty-five turns, and an average of 3,000 tokens of tool output added per turn. By the final turn the transcript is around 87,000 tokens, and the cumulative input across the run is roughly 1.2 million tokens.

Uncached, that is about $3.60 of input. Output at 15,000 tokens adds about $0.23. So roughly $3.83 per run, of which 94 percent is input.

Now enable prompt caching on the stable prefix. If the system prompt and the older portion of the transcript stay cached, the majority of that input bills at $0.30 rather than $3, and the run falls to well under a dollar. Nothing about the agent changed.

Which levers actually move

Ranked by effect, which is not the order most teams try them in.

Prompt caching. Because input dominates and the transcript prefix is highly repetitive across turns, this is the single largest lever available. An order-of-magnitude discount on the majority of your tokens is not something a model swap can match. Prompt caching savings math covers structuring for it.

Tool output size. Every token a tool returns is paid for on that turn and on every subsequent turn. A tool returning 8,000 tokens on turn five costs that 8,000 twenty more times over a twenty-five-turn run. Trimming tool output is worth roughly its size multiplied by the remaining turn count.

Turn count. Because cost is quadratic in turns, halving turns cuts cost by roughly three quarters rather than half. Better tools and clearer instructions that get the agent to the answer in fewer steps pay off disproportionately.

Compaction. Summarising completed work and dropping raw output caps transcript growth. Note the interaction with caching — compaction invalidates the cached prefix from the rewrite point, so compact on a threshold rather than continuously. Context compaction strategies covers the trade.

Model choice. Genuinely last. A cheaper model that needs more turns can cost more in total, because turns are quadratic and price is linear.

The measurement to build

Log per run: total input tokens, cached input tokens, output tokens, turn count, and whether the run succeeded.

Then compute cost per successful run rather than cost per run. Failed runs consume tokens and produce nothing, so a model or configuration with a lower per-run cost and a worse success rate can be more expensive per unit of finished work. This is the number that should drive decisions and it is rarely the one on the dashboard. The hidden cost of failed runs covers the accounting.

Also record the distribution rather than the mean. Agent cost is heavily skewed — most runs are cheap, a few are enormous. A mean hides the tail that is actually driving your bill, and the tail is usually where the fixable problem lives.

What good looks like

A healthy agent workload has a high cached-input fraction, a turn distribution with a short tail, tool outputs measured in hundreds rather than thousands of tokens, and a cost-per-successful-run figure that is stable week to week.

If any of those is off, fix it before comparing models. The gap between a well-configured agent and a badly configured one on the same model is usually larger than the gap between two models on the same configuration.