Scaling Laws Explained: What Predicts a Model Getting Better

Scaling laws are the empirical finding that a language model's training loss falls smoothly and predictably as you increase model size, training data and compute together. They are the reason the last several years of model releases looked less like research surprises and more like a construction schedule.

They are also routinely overstated. Understanding what they claim, and what they conspicuously do not, explains a lot about why 2026 releases look the way they do.

What the finding actually is

If you train a family of models across a wide range of sizes and data budgets, and plot the resulting loss against compute, the points fall on a strikingly regular curve rather than scattering.

That regularity is the whole result. It means you can train a set of small models cheaply, fit the curve, and predict what a much larger training run will achieve before committing to it. For a run costing an enormous amount of compute, that predictability is the difference between an engineering project and a gamble.

Two properties matter. The curve has diminishing returns — each further reduction in loss costs disproportionately more — and it does not level off within the range that has been tested. Progress keeps arriving; it just gets steadily more expensive.

The correction that changed everything

Early scaling work was read as saying to make models as large as possible. Labs did, and trained them on relatively modest amounts of data.

The later correction showed those models were badly undertrained. For a fixed compute budget, the loss-optimal configuration uses far more training data relative to parameters than had been assumed. A smaller model trained on much more data beats a larger model trained on less, at the same total cost.

This reoriented the field. Parameter counts stopped being the headline, data quality and quantity became the constraint, and the practical consequence is that modern models of a given size are dramatically more capable than models of the same size from a few years earlier. Model size versus capability covers why the count stopped working as a proxy.

There is a second correction on top of it. The loss-optimal configuration is not the deployment-optimal one. A model you will serve billions of times is worth overtraining well past the point of compute efficiency, because inference cost is paid forever and training cost is paid once. That is why small dense models keep getting better than their size suggests.

What the curve does not promise

Scaling laws predict loss on the training objective. Loss is next-token prediction accuracy, averaged over a held-out sample. It is not a measure of anything you care about.

The relationship between loss and useful capability is real but loose and nonlinear. Specific abilities appear at different points, sometimes abruptly enough that people describe them as emergent, and no scaling law tells you where those thresholds sit for a given skill.

Nor do the curves say anything about instruction following, refusal behaviour, tool-calling reliability or output formatting. All of those come from post-training on top of a pretrained base, and two labs starting from comparably scaled bases can ship models that feel entirely different. How LLMs actually work covers where that split sits.

They also say nothing about factual reliability. A lower-loss model hallucinates in more plausible ways, not obviously less often, because the objective rewards fluent continuation rather than truth. Why LLMs hallucinate covers that mismatch.

Data became the binding constraint

The correction made data the scarce input, and high-quality text is finite in a way compute is not.

Labs responded by improving the mixture rather than simply adding volume: heavier filtering, deduplication, more code, more curated technical material, and large quantities of synthetic data generated and verified by existing models.

This is why two models trained with similar compute can differ so much. The scaling curve assumes a fixed data distribution; change the distribution and you move to a different curve entirely. Data mixture is now among the most closely held details of any frontier training run, and it is a large part of why released model cards say so little about it.

It also explains why open-weight models can compete closely with closed ones at the frontier. The techniques are broadly known, and the advantage sits in execution and data rather than in a secret architecture. Open weight versus closed models covers where the remaining gaps are.

The axis that moved to inference

The most visible change in 2026 is that a large share of capability gains no longer come from the training curve at all.

Reasoning models spend additional computation at inference time, generating intermediate reasoning before answering. Performance improves with that spend, along a separate curve from the training one, and it is a curve you control at request time rather than one the lab fixed months ago.

GLM-5.2 exposing two reasoning effort levels is this made explicit — the same weights, two points on the inference-compute curve, chosen per request. Reasoning models explained covers how that spend behaves.

The practical consequence is that your cost model changed. Training compute is the vendor's problem; inference compute is yours, and it is now a capability dial rather than a fixed property of the model. Inference versus training cost covers how those two budgets differ.

What this means for choosing models

Stop reading capability off size, and stop assuming a newer generation is uniformly better. Improvements are unevenly distributed across skills, so a release that gains on agentic coding may be flat or worse on the specific thing you use it for.

Expect the pace of general improvement to continue while its cost rises, which means the gap between frontier and merely good models narrows over time for most everyday work. A model that was frontier eighteen months ago is now cheap, and often sufficient.

The operational takeaway is to re-evaluate on your own tasks each time you consider switching, rather than trusting the trend line. Benchmarking LLMs on your own work covers doing that in an afternoon.