Tool Call Retries: Retry the Transport, Not the Judgement

An agent that hits a failing tool call will try again. That is usually the right instinct and usually implemented as a single retry policy applied to everything, which means the loop treats a network blip and a wrong argument as the same event.

They are not the same event, and they need opposite responses. One should be retried identically and invisibly. The other should never be retried identically, because the identical call will fail identically and you will pay for the privilege three times.

Two kinds of failure

Transport failures are failures of the mechanism. A timeout, a connection reset, a 500, a 429. The call was well-formed and the world was temporarily unable to serve it. Repeating exactly the same call later is the correct fix, and the model does not need to know it happened.

Semantic failures are failures of judgement. A path that does not exist, a parameter out of range, a query with a syntax error, a permission the agent does not have. The call was delivered and understood, and the answer is no. Repeating it changes nothing.

Almost every retry pathology comes from handling the second category with machinery designed for the first. Classify before you retry, and the rest of the design follows. The API error code reference covers which status codes fall where.

Transport: retry in the executor, silently

Transport failures should be absorbed below the loop. The tool retries internally with exponential backoff and jitter, and the model only ever sees the eventual result — success, or a clear terminal error after the budget is exhausted.

Keep the model out of it because every failed attempt that reaches the transcript costs tokens on this turn and every turn after. A retry the agent never sees costs a few hundred milliseconds; a retry it does see costs context for the rest of the session.

Respect the server when it tells you something. A 429 with a Retry-After header is an instruction, not a suggestion, and honouring it is faster than backing off blindly. Rate limits and retries covers the backoff shapes worth using.

Semantic: change something or stop

When the call was wrong, the only useful retry is a different call. The loop's job is to give the model enough information to make it different.

That means the error message has to be actionable. "Not found" leads to an identical retry. "No such file: src/uti1s.ts. Nearest match in this directory: src/utils.ts" leads to a corrected call almost every time. The error message is the highest-leverage text in an agent, because the model reads it precisely when it needs to change behaviour.

Where a semantic failure repeats despite a good message, that is signal, not noise. Two identical failures in a row means the model is stuck and a third attempt is very unlikely to differ. Agent error recovery patterns covers the escalation ladder from there.

Detect the repeat before you pay for it

The cheapest guard in an agent loop is a normalised signature of each tool call — tool name plus canonicalised arguments — kept in a set for the session.

When the same signature fails twice, stop treating it as a retry and intervene. Inject a message naming what was tried and instructing the model to take a different approach. When it fails a third time, disable that avenue or hand off.

Without this guard, a stuck agent will happily burn its entire turn budget on one call, and the transcript fills with near-identical failures that make everything after them worse. Detecting agent loops covers the detection in detail.

Retries need a budget, not just a count

"Three attempts" is the usual policy and it is the wrong unit. Three attempts at a cheap local call is nothing; three attempts at a long agent turn with a full transcript is real money.

Budget retries in tokens or currency alongside attempts, and enforce both. A session that has spent its retry allowance should degrade deliberately — fall back to a simpler path, or stop and report — rather than continuing to spend at the same rate.

This is one of the largest and least visible line items in agent spend, because retries do not appear as a separate category on any invoice. The hidden cost of retries works through the arithmetic.

Make retries safe to perform

Retrying only works if a repeated call is harmless. For reads that is automatic. For anything that writes, sends or charges, it has to be designed.

Idempotent operations — where calling twice has the same effect as calling once — remove the whole category. Where true idempotency is impossible, accept an operation identifier so the tool can recognise a duplicate and return the original result rather than performing the action again.

This matters most exactly where you would least like it to go wrong. A repeated file read is waste; a repeated deployment, payment or email is an incident. Tool schema design covers building tools that tolerate this.

What to instrument

Log every retry with its classification, the tool, the attempt number and the outcome. Three numbers then become visible that are otherwise invisible: retry rate per tool, the share of retries that eventually succeed, and the token cost of retries as a fraction of total spend.

A tool with a high retry rate and a low eventual-success rate is not flaky, it is badly specified — the model keeps calling it wrong, and the fix is in the description or the schema. A tool with a high retry rate and a high success rate is genuinely flaky infrastructure, and belongs behind silent transport retries.

The two look identical on a dashboard that only counts retries. Separating them is what turns retry data into a fix rather than an alert.