Designing an AI Spend Dashboard People Actually Read

The default LLM cost dashboard is a big total-tokens number, a stacked area chart by model, and a table of requests. It gets built in an afternoon, looks impressive in a review, and nobody opens it again because it never answers a question anyone has.

A dashboard earns its place when someone who glances at it changes what they do. That is a high bar and it rules out most of what usually ends up on the screen. The useful version is smaller than the default one.

Raw token counts are the wrong top-line

Tokens are a metering unit. They are the right primitive to log and the wrong thing to put at the top of a page, for three reasons.

They are not comparable across models, because a token from one model at one price is not a token from another at four times the price. A dashboard headline showing total tokens falling while spend rises is not a bug in the data; it is the metric failing at its job. Input and output tokens are also priced differently and usually differ by an order of magnitude in volume, so summing them produces a number with no meaning at all.

And tokens do not carry a sense of scale for anyone outside the team that built the system. A person who sees four billion tokens has no idea whether that is a problem. A person who sees a cost per pull request that doubled since March knows immediately. Input versus output token pricing covers why the two streams have to be tracked separately even when you are not displaying them.

Keep tokens on the page — they are how you diagnose a change — but keep them below the fold, split by input, output and cached.

The panels worth keeping

Five, and the discipline is in refusing the sixth.

Spend rate over time, by workload. Cost per day or per hour, stacked by tagged workload, with a comparable previous period overlaid. This is the panel people actually look at, and the workload split is what turns "spend rose" into a statement about a specific system.

Cost per unit of work. One line per major workload — cost per agent run, per review, per processed document. This is the panel that distinguishes growth from regression and it is the one most dashboards omit.

Cached input fraction. The single most sensitive early indicator of a prompt-structure regression, and one that moves the bill severalfold with nothing else visibly changing. A cliff in this line is nearly always a deploy. Prompt caching savings math covers the arithmetic.

Model mix by spend. Not by request count. A small share of requests on an expensive tier can dominate the bill, and mix drift — a fallback route firing more often, a default quietly changing — is invisible in every other panel.

Failure and retry share. Tokens spent on attempts that produced nothing. This is money with a zero return and it is usually the easiest thing on the page to reduce. The hidden cost of retries covers the accounting.

Latency and cost on the same view

Cost and latency are usually owned by different dashboards and different people, which guarantees that every optimisation on one side surprises the other.

Nearly every cost lever moves latency. Routing to a smaller model cuts spend and usually cuts time to completion. Trimming context cuts spend and cuts time to first token. But raising retry limits improves success rate at the cost of both, batching cuts unit cost while increasing latency, and switching to a reasoning-heavy model can improve first-attempt success while multiplying both output tokens and wall-clock time.

Put p50 and p95 latency per workload directly beside cost per unit of work for the same workload. The pairing makes the trade-off visible at the moment somebody is considering it, rather than in a postmortem two weeks later. Latency-adjusted model scoring covers how to weigh the two when choosing a model.

Per-workload beats per-model

The instinct is to break everything down by model, because that is the dimension the provider gives you for free. It is the least useful cut available.

Knowing that a particular model accounts for most of your spend does not tell you what to do, because the model is a consequence of decisions made inside workloads. Knowing that the nightly regression suite accounts for most of your spend tells you exactly where to look, and whether the suite should run nightly at all.

Model breakdowns are worth one panel, as a mix-drift detector. Everything else should be cut by workload first, with model available as a drill-down. That requires the tag to be attached at the call site and propagated, which is the same prerequisite as attribution generally. Agent cost attribution covers the plumbing.

Vanity panels to leave out

A cumulative spend curve that only goes up. It is always rising, it always looks alarming, and it conveys nothing that the rate panel does not convey better.

Total request count. Requests are not a unit of work — one task can be forty of them — so the number moves for reasons unrelated to anything you would act on.

A leaderboard of the highest-spending individuals. It produces defensive behaviour and suppresses exactly the experimentation you want, and the top of the list is usually whoever is doing the most valuable work.

Tokens per second and other throughput figures on a cost dashboard. They belong on a performance dashboard; here they are decoration.

And any panel with a target line that nobody agreed to. A threshold on a chart implies a commitment, and an unowned one gets crossed, ignored, and teaches readers that the whole page is advisory.

Two audiences, two pages

The engineer debugging a spike and the manager reviewing a budget want incompatible things, and trying to serve both on one screen produces a page that serves neither.

The operational page is high-resolution and short-window: hourly spend rate, per-workload unit cost, cached fraction, retry share, latency percentiles, and a link straight from any spike into the traces behind it. Default range of a few days.

The review page is low-resolution and long-window: monthly spend by team and workload, unit metrics with six months of history, model mix, and written commentary on the largest movers. Default range of two quarters. That page is the input to a forecast, which is a different exercise again. Forecasting team AI spend covers it.

A test for every panel

Before adding anything, answer two questions. What decision does this panel change, and who makes that decision? If either answer is vague, the panel is decoration, and decoration is not neutral — it raises the cost of reading the page and makes the panels that matter harder to find.

Then apply the same test to what is already there, once a quarter. Dashboards accumulate. The version that gets read is the one somebody has been willing to delete from.