The Token Cost of Tool Calls: Definitions Bill Every Turn

Tool calling is usually costed as though the tools were free and only the model's reasoning were billed. In practice tools contribute on both sides of every request, and in a long agent run they frequently dominate.

Definitions are sent every turn

Your tool schemas are part of the prompt. They are serialised, included in the request, and billed as input tokens — on every single turn, not once per session.

A set of fifteen tools with detailed parameter descriptions can run to 4,000 tokens. Across a twenty-five turn session that is 100,000 input tokens spent purely on describing what the tools do, before any actual work.

This is why tool count matters for cost and not only for reliability. Merging four overlapping search tools into one does not just reduce wrong-tool errors — it removes their definitions from every request for the life of the system. Tool schema design covers doing it well.

The good news is that definitions are stable, so they sit in the cacheable prefix. With prompt caching enabled they bill at a fraction of the standard rate, which is a strong argument for placing them early in the prompt and never varying them per request.

Results are the bigger half

Whatever a tool returns enters the transcript and stays there. It is billed on the turn it arrives and on every subsequent turn of the run.

The multiplier is what makes this severe. A tool returning 6,000 tokens on turn four of a twenty-four turn run does not cost 6,000 tokens. It costs roughly 6,000 multiplied by the twenty turns that follow — around 120,000 tokens for one call.

Put concretely: a single verbose file read early in a session can cost more than all of the model's generated output for the entire run.

What to return instead

Design tool output for the model's next decision rather than for completeness.

A search should return paths, line numbers and a short excerpt — enough to decide what to read next — not full file contents for every match. A file reader should accept a line range. A test runner should return failures with their assertions, not the entire log including every passing case.

Where output could be large, truncate with an explicit marker and let the model request more. A result that says it showed the first fifty of four hundred matches is more useful than four hundred matches, because the model can narrow the search rather than wading through them.

Set a hard per-tool ceiling in code. Without one, a tool that usually returns 200 tokens will eventually return 90,000 when it hits an unusual input, and that single event can dominate a run's cost. Tool result formatting covers the patterns.

Failed calls are pure waste

A malformed tool call costs the tokens to generate it, the tokens of the error returned, and the tokens of the corrected retry — and all three stay in the transcript for the rest of the run.

So tool-calling reliability is a cost issue, not only a correctness one. A model or schema producing a five percent invalid-call rate is adding meaningfully to spend on top of the failures themselves.

This is another reason error messages should be specific. A vague error produces an identical retry and another round of waste; a specific one produces a correction. Best model for tool calling covers measuring the rate.

Working the numbers for your own setup

Instrument three counts per run: tokens spent on tool definitions, tokens returned by tool results, and tokens generated by the model. Most teams find the middle number is several times the third.

Then rank your tools by total tokens returned across a sample of runs. The distribution is usually extremely skewed — one or two tools account for most of the volume, and they are almost always the ones returning file contents or logs.

Fixing those two tools is a contained afternoon of work that typically cuts agent cost by a large fraction, and it improves quality at the same time by leaving more attention for material that matters.

The rule to keep

Every token a tool emits is paid for once per remaining turn. Costing a tool by what it returns in isolation understates it by roughly the number of turns left in the run, which is why verbose tools feel cheap in testing and are expensive in production.