Escalation Rate: The Metric That Decides Your Default Model

Two-tier model setups are sold on an appealing arithmetic: run the cheap model by default, fall back to the expensive one when it struggles, and pay the premium rate only on the hard minority. The saving looks obvious.

It is obvious only if the minority is genuinely a minority. There is a single number that decides whether the arrangement saves money or quietly costs more than using the expensive model outright, and most teams running a cascade have never measured it.

What escalation rate actually is

Escalation rate is the share of tasks that start on your cheap tier and end up being retried on the expensive one. Not the share that fail — the share that fail and then get redone.

The distinction matters because an escalated task is paid for twice. You spent the cheap model's tokens discovering it could not finish, then spent the expensive model's tokens doing the work. Both appear on the bill.

That makes escalation rate different in kind from a quality metric. Accuracy tells you how often the cheap model is right; escalation rate tells you how often being wrong costs you a second full run. Those numbers diverge whenever failures are expensive to detect, which is most of the time in agent work.

The break-even is lower than people assume

Work it through with your own figures. Let the cheap model cost c per task and the expensive one cost e. Running everything on the expensive tier costs e. Running the cascade costs c on every task, plus e on the escalated share r — so c + r·e.

The cascade wins while c + r·e < e, which rearranges to r < 1 − c/e. If the expensive model costs five times the cheap one, the cascade stops paying once escalation passes 80%. That sounds like enormous headroom.

It shrinks fast when the ratio is narrower. At a 2× price gap the break-even is 50%; at 1.5× it is 33%. And the price gap between a mid-tier and a frontier model is often much closer to 2× than to 5×, which is why cascades that looked free on a spreadsheet turn out to be marginal in production.

This is also why the arithmetic in when a cheap model is enough depends on knowing the ratio before you design the routing, not after.

The costs the simple formula leaves out

The formula above is optimistic. An escalated task does not just cost two model calls — it costs the wall-clock time of the first attempt, which in an interactive workflow is time a developer spent waiting for an answer that was thrown away.

It also costs whatever the failed attempt did before failing. A cheap model that half-completes a refactor and then gets escalated leaves the expensive model reading a partially modified repository, which is a harder problem than the original. Some escalations are more expensive than the task would have been on the expensive tier from the start.

And detection is not free. If you determine escalation by running tests, you pay for the test run on every task, not just the failing ones. If you determine it with a judge model, you pay for the judge. The hidden cost of retries covers this compounding in more detail.

Measuring it without building anything

You need three fields per task, and you almost certainly already log two of them: which model handled it, and whether it succeeded. The third is a task identifier that survives the retry, so a second attempt can be linked to its first.

That correlation id is the piece teams usually miss. Without it, a retry looks like an unrelated task in the logs, escalation rate reads as zero, and the cascade appears to be working perfectly. Adding one field to your request logging is the whole implementation.

Then compute the rate weekly, split by workload rather than in aggregate. An overall figure of 20% can easily be 5% on code review and 60% on multi-file refactoring, and those two numbers imply opposite routing decisions.

Reading the number you get

Below roughly 10%, the cheap tier is doing its job and the premium model is a rounding error in your bill. Leave the routing alone and spend your attention elsewhere.

Between 10% and the break-even you computed, the cascade is working but is worth tuning. The usual lever is the trigger rather than the models: escalating on a mechanical signal such as file count or a failed verification gate wastes fewer first attempts than escalating on a judge's opinion after the fact.

Above the break-even, stop. Promote the expensive model to default and use the cheap one only for the specific narrow tasks where it demonstrably succeeds. A cascade running above its break-even is not a cost optimisation, it is a tax you are paying for the appearance of one.

Why the rate moves on its own

Escalation rate is not a constant you measure once. It drifts, and the drivers are mostly things you did rather than things the model did.

Growing prompts push it up, because a longer context degrades the cheaper model faster than the stronger one — the effect described in the lost-in-the-middle problem. Task mix shifts push it up too, as teams that get comfortable with a tool start giving it harder work.

Model upgrades move it in either direction and are the most commonly missed cause. A new version of your cheap tier can halve escalation, making a cascade that was marginal suddenly worthwhile; a new version of the expensive tier changes the price ratio and therefore the break-even itself. Recompute after any model upgrade, not just after a routing change.

The takeaway

Log a correlation id, compute the escalated share per workload, and compare it against 1 − c/e for your actual price ratio. That single comparison tells you whether your default model is the right one, and it is cheaper to run than any benchmark.

If you have never measured it, assume the number is higher than you would guess. Cascades fail quietly, because the failure mode looks exactly like the system working.