The Self-Hosting Break-Even Point, With Honest Inputs

The self-hosting calculation is usually presented as GPU rental against token spend. Run the numbers that way and self-hosting looks compelling at surprisingly modest volume, which is why so many teams start down the path and abandon it.

The comparison is not wrong. It is missing terms, and the missing terms are large enough to move the break-even point by an order of magnitude.

The naive version

Take a workload of 200 million input and 20 million output tokens a month, on DeepSeek V4 Pro at around $0.44 and $0.87 per million. That is roughly $88 plus $17, so about $105 a month.

Against that, a rented accelerator running a model you can actually host. Whatever your hourly rate, multiply by 730 hours and compare.

At that volume the API almost certainly wins. Push volume up tenfold and the comparison narrows, because API cost scales linearly while the GPU cost is fixed. Somewhere there is a crossing point, and the naive calculation finds it.

The terms that move it

Engineering time. Someone builds the deployment, keeps the serving stack current, handles out-of-memory failures, upgrades models, and is on call when it breaks. Even at a modest fraction of one engineer, this frequently exceeds the hardware cost outright — and it is the single most commonly omitted term.

Idle capacity. A rented GPU costs the same at 3am on a Sunday. If your traffic is concentrated in working hours, you may be paying for three times the capacity you use. Per-token pricing charges nothing when nobody is asking, which is a genuine structural advantage that the fixed-cost comparison ignores.

Redundancy. One accelerator is a single point of failure. If the workload matters, you need at least two, which doubles the hardware side before any traffic is served.

Throughput under real load. Single-request benchmarks flatter self-hosting badly. At real concurrency, batching behaviour, KV cache memory pressure and queueing all reduce effective capacity. Measure at your actual concurrency or you will size the deployment wrong. KV cache explained covers why long-context sessions consume memory per session.

Model churn. New models arrive constantly. Each is a redeployment and a re-evaluation, and staying current is ongoing work rather than a one-time setup.

Which model you can host changes the comparison

This is frequently overlooked. Self-hosting does not give you the same model more cheaply — it gives you a different, usually smaller model.

Kimi K3's weights are roughly 1.6TB, which is a multi-node deployment. DeepSeek V4 Pro holds 1.6T total parameters and needs memory for all of them despite activating about 49B per token. Neither is a single-GPU proposition.

What does fit on one accelerator is a dense model like Qwen 3.6 27B, which reports 77.2 percent on SWE-bench Verified. That is genuinely strong, and it is not the same as what you were using through the API.

So the honest comparison is not "same capability, lower cost". It is "somewhat less capability, different cost structure, plus operational burden". Whether that trade is good depends on whether the smaller model clears your quality bar, which is a measurement rather than an assumption. Best model for self-hosting covers the options.

When the calculation is irrelevant

Some constraints decide it regardless of cost.

Data residency where tokens legally cannot leave your network. Air-gapped environments with no reachable API. Regulatory requirements about where inference happens. In all of these the question is not whether self-hosting is cheaper but which model fits on hardware you are permitted to use.

Conversely, if your volume is low and variable, per-token pricing is structurally better and no amount of optimisation changes that. Paying only for what you use is exactly right for a workload that is mostly idle.

How to actually decide

Run on an API for a month first, with proper instrumentation. You need three things you cannot guess: real monthly token volume, the shape of the traffic across the day, and whether a smaller model clears your quality bar on your own tasks.

Then compute break-even including engineering time at a realistic fraction of a salary, redundancy, and utilisation based on your measured traffic shape rather than an assumed steady rate.

Most teams who do this discover they are well below the crossing point, and that the answer changes only at volumes they do not currently have. That is a useful thing to learn cheaply rather than after building a deployment. Self-hosting vs managed API cost covers the operational side in more depth.