Model Selection for Cost Ceilings: Working Backwards From a Budget

Most model selection advice assumes you pick the model and then discover the bill. A large number of real decisions run the other way: finance sets a number, and the job is to build something good inside it.

That inversion changes the method entirely. The question stops being which model is best and becomes which models are affordable at your volume, and what quality is reachable within that set.

Turn the budget into tokens per request

A monthly figure is not actionable. Convert it into an allowance per unit of work, because that is the number an engineer can design against.

Start with expected volume: requests per month, or pull requests, or developer-days. Divide the budget by it to get spend per unit. Then, using a candidate model's prices, convert that into a token allowance. If you have $0.05 per request and a model charges $3 per million input tokens and $15 per million output, an all-input request buys roughly 16,000 tokens; a request with 2,000 output tokens leaves about 6,000 for input.

Do that arithmetic for each candidate before comparing anything else. It usually eliminates half the shortlist immediately, and it tells you something more useful than a ranking: how much context you can afford to send.

Leave headroom. Real traffic has a tail, retries happen, and a budget computed against the mean is exceeded in the first week. Plan against p90 request size, not the average. Spend forecasting covers building the projection.

The input/output ratio decides which model is cheap

Headline prices are misleading because the ratio between input and output pricing differs sharply across models, and your workload has its own ratio.

Kimi K3 lists $3 in and $15 out per million — a five-to-one spread. DeepSeek V4 Pro lists roughly $0.44 in and $0.87 out, about two-to-one. DeepSeek V4 Flash sits near $0.14 in and $0.28 out. GLM-5.2 sits around $1.40 in and $4.40 out.

A workload that reads a large amount of code and emits a short verdict is dominated by input pricing. An agent that generates long patches is dominated by output. The same two models can swap ranking between those cases, which is why a blended per-million figure is the wrong comparison unit. Input versus output token pricing covers the asymmetry.

Pull your actual ratio from logs. It is one query and it frequently overturns a preference formed from a price page.

Caching moves the ceiling more than model choice

Before switching to a cheaper model, check whether your prompts share a long stable prefix. If they do, cached input pricing is a larger lever than almost any substitution.

Kimi K3 lists cached input at $0.30 against $3 uncached — a tenfold reduction on the cached portion. For a workload with a large fixed system prompt, tool schemas and a repeated document set, that can cut the input bill dramatically without changing model, prompt or quality.

The requirement is prefix stability. Anything that varies early in the prompt — an injected timestamp, a user identifier, a reordered tool list — invalidates the cache for everything after it. Restructuring so the stable content comes first is often a half-day change with a large payoff. Prompt caching savings math covers estimating the hit rate.

Reduce what you send before you reduce what you pay for it

The cheapest token is the one not sent. Teams under a ceiling reach for a cheaper model first and context discipline second, which is the wrong order.

Sending an entire repository when three files are relevant is the most common form of waste, and it is expensive twice: once in input tokens and once in quality, because relevant content gets diluted. Reducing token usage covers the practical reductions.

Conversation history is the other large item. In an agent loop the full transcript is re-sent every step, so a twenty-step task pays for early messages twenty times. Compaction bounds that growth. Context compaction strategies covers doing it without losing the thread.

Both of these reduce cost without reducing capability, which makes them strictly better than moving down a model tier. Exhaust them first.

Choose the model last, and route the rest

With volume, ratio, caching and context discipline settled, model choice becomes a narrow question: which model clears your quality floor within the remaining allowance.

Rank candidates on cost per successful outcome rather than per-token price, because a cheap model that needs two attempts is not cheap. Cost-adjusted scoring covers the measurement.

Then split the workload rather than picking one compromise. Sending the trivial majority to a cheap model and reserving the expensive one for genuinely hard requests keeps average cost inside the ceiling while preserving quality where it is visible. Multi-model strategies covers the split.

Enforce the ceiling in code

A budget that exists only in a spreadsheet is discovered at the end of the month. Put it in the request path.

Set explicit maximum output tokens on every call. It is the simplest and most effective guard, and it bounds the most expensive half of the bill directly. Set input caps too, and truncate or refuse rather than silently sending an enormous prompt.

Cap agent iterations. An unbounded loop is the single largest source of surprise spend, because cost scales with steps and steps are not bounded by anything the model is aware of. Agent token budgets covers enforcing a per-task allowance.

Finally, alert on the daily run rate rather than the monthly total. A month is too long a feedback loop; a daily figure compared against the budget divided by thirty catches an overrun on day two instead of day twenty-eight.