The Token Cost of Reasoning Models: Paying for Hidden Output

A reasoning model works through a problem before answering. That intermediate work is generated text, billed at output rates, and frequently invisible in the response you receive.

Teams costing these models from the visible answer length underestimate by a wide margin, and the error is worst on exactly the hard problems where reasoning models are chosen.

Why the hidden tokens are expensive

Output rates are typically several times input rates. Kimi K3 lists $15 per million output against $3 input — a five-fold difference.

Reasoning tokens are output tokens. So a model that emits 8,000 tokens of internal reasoning before a 500-token answer has generated 8,500 output tokens, and you are billed for all of them at the higher rate. The visible response is six percent of what you paid for.

That ratio is not unusual on genuinely hard problems. It is also highly variable — the same model on an easy question may reason briefly, and on a hard one may reason at length. Cost per request therefore has a much wider distribution than with a non-reasoning model, which makes budgeting harder rather than just more expensive.

Where it is worth paying

The mechanism explains the answer. Computation per token is fixed, so a problem needing more computation than one forward pass allows cannot be solved in a single token. Intermediate reasoning spreads the computation across many passes. Chain of thought explained covers this properly.

That means reasoning helps where the task has sequential dependency — arithmetic, multi-premise deduction, code tracing, planning. It helps least on retrieval and pattern-matching tasks, where a knowledgeable person would answer immediately.

So the expensive mode is worth it on the second kind of problem and pure waste on the first. Applying it uniformly across a pipeline means paying the premium on the easy majority to benefit the hard minority.

Effort levels are the practical control

Several models expose a reasoning effort setting. GLM-5.2 offers two levels, high and xhigh.

This is the most useful cost lever available on a reasoning model, because it works inside a single integration. Route easy requests to low effort and hard ones to high, and you capture much of the benefit of switching models without maintaining two.

The classification does not need to be sophisticated. Task type, input length, or an explicit caller-supplied hint all work. The gain comes from not spending maximum effort on everything, and almost any reasonable split achieves that.

Measure the ratio for your workload

Most providers report reasoning tokens separately in the usage payload. Log them.

Compute the ratio of reasoning tokens to visible output tokens across a representative sample. If it is low, reasoning is adding little and you may be paying for a mode you do not need. If it is very high on tasks you consider easy, your prompt is inviting deliberation the task does not require.

Look at the distribution rather than the mean, because reasoning length is heavily skewed. A small number of requests where the model reasons at extreme length can dominate spend, and those are usually worth investigating individually — they are frequently ambiguous prompts rather than hard problems.

The comparison that decides it

Run your evaluation set twice: once with reasoning enabled or at high effort, once without or at low effort. Record accuracy and total tokens for each.

Three outcomes are common. Accuracy unchanged means you are paying for nothing and should turn it down. Accuracy improved on a subset means you should route rather than apply it globally. Accuracy improved across the board means the workload genuinely needs it, which is rarer than teams assume.

The second outcome is the usual one, and it is the one that justifies the engineering effort of classification. When a cheap model is enough covers setting the threshold.

Cap the output

Whatever you conclude, set an explicit maximum output length. A reasoning model given an ambiguous prompt can reason for a very long time, and without a ceiling a single malformed request can cost more than a hundred normal ones.

This matters more with models offering large maximum outputs — GLM-5.2's 128K ceiling is genuinely useful for generation work and is also a lot of tokens to emit by accident. Cap it deliberately rather than relying on the model to stop.