Best Model for Batch Jobs: Throughput, Not Intelligence

Batch work removes the constraint that shapes most model selection. Nobody is watching a spinner. A job that classifies four hundred thousand support tickets overnight does not care whether each call takes two seconds or twenty.

Removing latency from the equation changes the answer substantially, and usually in the direction of a cheaper model than you would pick for interactive use.

Unit economics are the whole decision

At batch scale, price per million tokens stops being a footnote and becomes the entire budget line. The difference between tiers that looks trivial on one request is decisive across a million.

Take a job of 500,000 items at 1,200 input and 300 output tokens each: 600M input and 150M output. On DeepSeek V4 Flash at roughly $0.14 in and $0.28 out per million, that is around $126. On Kimi K3 at $3 in and $15 out, the same job is around $4,050.

That is not a preference, it is a different project. A $126 job runs whenever someone thinks it might be useful; a $4,000 job needs approval and a business case. The cheap tier does not merely save money, it changes what you are willing to attempt.

Which means the real question is not which model is best. It is the cheapest model whose accuracy clears your threshold — and for classification, extraction, tagging and summarisation, that is very often the cheap tier. When a cheap model is enough covers where the line sits.

Measure accuracy on a sample before you commit

The correct process is boring and almost nobody follows it. Take two hundred items, label them by hand, and run every candidate model against that set.

What you get is an accuracy figure per model, at a known cost per item. Now the decision is arithmetic rather than opinion. If Flash hits 94 percent and K3 hits 97 percent, you can ask directly whether three points is worth thirty times the price for this particular job — and for ticket routing it is not, while for anything that feeds a financial report it may be.

The sample also tells you the shape of the failures, which matters more than the rate. Errors concentrated in one identifiable category can be routed to a stronger model; errors scattered randomly cannot, and force you up a tier.

Two hundred labelled items is an afternoon of work that routinely saves thousands. Benchmarking models on your own work covers building the harness once so you can reuse it per job.

Cascade instead of choosing

The setup that usually wins is not one model but two, arranged so the cheap one handles the volume.

Run everything through the cheap model first and require it to return a confidence signal alongside its answer. Route only the low-confidence cases to a stronger model. If 90 percent of items come back confident, you pay frontier prices on a tenth of the volume and get close to frontier accuracy overall.

The mechanism that makes this work is asking for structured output with an explicit uncertainty field, and — importantly — validating that the model actually uses it rather than marking everything confident. Some models are badly calibrated and will claim certainty uniformly, which collapses the cascade back into a single-tier system with extra steps.

Check calibration on your labelled sample: among the items the cheap model called confident, what was its actual accuracy? If that number is not clearly higher than its overall accuracy, the confidence signal is worthless and you need a different routing rule. Model routing and fallbacks covers the implementation.

Failure handling is where batch jobs actually break

A million-call job will encounter rate limits, timeouts, transient errors and occasional malformed output. The difference between a job that finishes and one that has to be restarted from zero is entirely in how you handle those.

Checkpoint aggressively. Write results as they arrive, keyed by item ID, so a crash at 780,000 items resumes rather than restarts. This is trivial to build and catastrophic to omit.

Make retries idempotent and bounded. A retry loop with no cap turns a provider outage into an enormous bill for nothing, and unbounded retries against a rate limit make the rate limit worse. Exponential backoff with a maximum attempt count, and a dead-letter queue for items that exhaust it. Rate limits and retries and the hidden cost of retries both cover this ground.

Validate output shape before storing. A malformed response caught at write time is an item to retry; the same response stored and discovered three weeks later is a data quality incident.

Exploit the batch shape

Batch work has structural properties that let you cut cost further, and they are easy to leave on the table.

Every item shares the same instructions, which is the ideal prompt caching scenario. Put the stable material first and the item-specific material last, and the shared prefix is billed at the cache rate. Kimi K3 prices cached input at $0.30 per million against $3, an order of magnitude on the part of the prompt that is identical across every call.

Concurrency is your throughput lever, since per-call latency is irrelevant. Find your provider's rate limit and run near it, with backoff for the overshoot. Going from four concurrent requests to sixty turns an overnight job into an hour.

And consider whether an item needs a model at all. Deterministic pre-filtering — dropping obvious cases with a regex or a rule before the model sees them — is free, and on many datasets it removes a large fraction of the volume. Reducing token usage covers the rest.

Recommendation

For most batch work, DeepSeek V4 Flash. At roughly $0.14 in and $0.28 out per million with a 1M context and an MIT licence, it is the default for classification, extraction, tagging and bulk summarisation.

Where accuracy on your sample does not clear the bar, DeepSeek V4 Pro at around $0.44 in and $0.87 out is the natural next step — same family, same licence, meaningfully stronger, still an order of magnitude below frontier pricing.

Reserve Kimi K3 for the low-confidence tail in a cascade rather than the whole run. The decision rule: label two hundred items, measure every candidate against them, and pick the cheapest model that clears your threshold — then cascade the rest.