Usage Caps and Overages: Limits That Do Not Break Work

A usage cap is easy to add and easy to get wrong. The naive version — one number, per account, hard stop — reliably converts a budget problem into an incident, usually at the worst possible time, because the moment you hit the cap is by definition the moment something unusual is happening.

A cap is not a number. It is a number plus a scope, a behaviour at the limit, and a warning path. Skip any of the three and you have built an outage generator with a finance justification.

Hard caps versus soft caps

A hard cap refuses requests once the threshold is reached. It gives you a guarantee: spend cannot exceed this figure, whatever your code does. That guarantee is worth having in exactly one situation, which is when you do not trust the thing consuming the budget — new agent code, a leaked key, a workload nobody owns.

A soft cap does not refuse anything. It fires an alert, escalates, and lets the work continue. It gives you a different guarantee: nothing breaks. That is the right shape when the workload is production-critical and the people who own it are reachable, because a human deciding to spend more is almost always a better outcome than a machine deciding to stop.

Most teams need both, at different levels. A soft cap at the point where spend has become notable and a hard cap far above it, positioned where the number stops being a budget overrun and starts being evidence of a bug. The gap between them is the space in which a person gets to make a decision, and that gap is the entire design.

Per-key, per-account, per-workload

Scope decides who suffers when the cap binds. An account-level cap means the runaway agent takes down the interactive users, the CI pipeline and the customer-facing feature simultaneously. It is the simplest cap to implement and the most destructive one to hit.

Per-key caps are usually the right granularity, because keys tend to map to workloads and workloads are what you actually want to isolate. Give the experimental agent its own key with a tight cap, the production pipeline its own key with a loose one, and a runaway in the first cannot touch the second. This is also why sharing one key across every tool is a bad default — it destroys the isolation before you get to use it.

Per-workload caps go further and are worth it once a single key serves multiple jobs. That requires attribution you may not have yet; attributing agent costs covers getting spend tagged at the level you want to limit it. Without attribution, per-key is as fine-grained as you can honestly enforce.

What should happen at the limit

Failing is the default and usually the worst of the three options. It surfaces as errors in whatever called you, and those errors are indistinguishable from a provider outage, so your on-call gets paged for a budget event. If you do fail, fail with a distinguishable error and a message that names the cap. Nobody should have to read your billing dashboard to work out why requests started returning 429.

Degrading is better where the work has a cheaper path. Route to a smaller model, drop optional enrichment steps, reduce the context you send, turn off the reasoning-heavy mode. Quality falls and the pipeline keeps moving, which for most batch work is the correct trade. Model routing and fallbacks covers the routing mechanics; the cap simply becomes another input to the routing decision.

Queueing is best where the work is genuinely deferrable. Hold requests until the window resets or a human approves an increase, then drain. It converts a spend spike into latency, which is the cheapest currency you have for anything asynchronous. It is also the option that fails silently if you do not bound the queue, so cap the queue depth too and shed with a clear error beyond it. Queueing and backpressure for agents goes through the mechanics.

Overage pricing shapes

If a vendor offers overage rather than a hard stop, the shape of the overage rate is worth as much attention as the base rate.

Flat overage charges the same unit rate above the allowance as below it. This is the honest shape and the easiest to model: your bill is simply linear in usage and the allowance is a prepayment rather than a discount. Punitive overage charges a premium above the allowance, sometimes a large one, which is a pricing structure designed to push you up a tier rather than to recover cost. Tiered overage steps you into the next plan automatically, which is fine if the step is small and unpleasant if it is a doubling.

The number to compute is the crossover: the usage level at which the current plan plus overage costs more than the next plan up. If you are consistently past that point you are paying a penalty for administrative inertia. If you cross it only occasionally, staying put and eating the overage is correct, and a spend forecast is what tells you which of those you are.

Ask one more question: does overage apply to the account pool or to each seat separately. Pooled overage absorbs an individual spike. Per-seat overage means one heavy user pays a premium while nine unused allowances expire — the worst arrangement available, and a common one. Per-seat versus per-token pricing covers why that combination is so bad.

Why a cap without a warning path is just an outage

This is the failure that keeps recurring. A team sets a monthly cap, forgets about it, and eleven months later a normal growth month collides with a number chosen when the workload was a quarter the size. Everything stops at 2pm on a Thursday and nobody knows why, because the cap is enforced somewhere nobody looks.

A cap needs a trajectory alert, not a threshold alert. Firing at 80 percent consumed tells you very little on day two of the month and very little on day twenty-eight. What you want is burn rate against time remaining: alert when the current daily rate projects to exceed the cap before the period ends. That fires early enough to act and does not fire during a normal month.

Attach the alert to a named owner and an escalation path with the authority to raise the cap. A cap that only an absent finance approver can lift is not a control, it is a scheduled outage. And log every rejection with the cap that caused it, so the post-mortem takes minutes rather than an afternoon of guessing. Alerting on token spikes covers the detection side in more depth.

Setting the numbers

Do not pick round numbers. Derive them from observed usage: take the peak day of the last quarter, multiply by the number of days in the period, and set the soft cap somewhere around that. It is deliberately loose, because a soft cap that fires during normal operation gets muted within a fortnight and then it is not a control at all.

Set the hard cap at a multiple of the soft cap large enough that reaching it means something is broken rather than busy. Three to five times is a reasonable starting point for most teams. You are not budgeting at that level — you are catching runaway loops, and a runaway loop overshoots by orders of magnitude, not by 20 percent.

Then review both quarterly against actual usage. Caps set once and never revisited are the direct cause of the Thursday-afternoon outage described above.

A checklist

Every cap should have a scope narrow enough that hitting it does not take down unrelated work, ideally per key. A defined behaviour at the limit that is deliberately chosen — fail, degrade or queue — rather than inherited from whatever your HTTP client does by default. A burn-rate alert that fires on trajectory rather than on a percentage consumed. A named owner who can raise it without a purchasing cycle. And a review date, because the correct cap for last quarter is rarely the correct cap for this one.