How large are open-weight model checkpoints?

Total parameters times bytes per parameter, plus a few percent of metadata. A dense 27B model is near 54 GB at 16-bit and 15 to 18 GB at 4-bit. Kimi K3, at 2.8T total parameters, ships full weights of approximately 1.6 TB.

Do mixture-of-experts models need less storage because fewer parameters are active?

No. Active parameters determine compute per token; total parameters determine disk and memory. All experts must be resident and stored, so a model activating 40B of 744B still occupies the full 744B on disk.

Is storage the main cost of self-hosting a model?

Almost never. Accelerator memory is the binding constraint, because the weights share HBM with the KV cache for every concurrent request. Size disk generously and spend the planning effort on the memory ceiling instead.

Planning Storage for Self-Hosted Model Weights

Teams planning their first self-hosted deployment usually budget disk for one model at one precision, discover three months later that they are holding nine checkpoints, and spend an afternoon working out which ones are safe to delete. The growth is predictable, and so is the fix.

Storage is also the cheapest resource in the stack, which is why the sizing conversation matters less than it feels like it should. The interesting part is not how many terabytes you need but how the layout affects cold-start time and rollback safety.

Checkpoint size is parameters times bytes per parameter

The arithmetic is genuinely that simple. Multiply the total parameter count by the number of bytes used to store each one, add a few percent for the tokeniser, config, index files and any duplicated shards.

At 16-bit precision that is two bytes per parameter. A dense 27B model such as Qwen 3.6 27B therefore lands near 54 GB. At 8-bit it is one byte, so the same model sits near 27 GB. At 4-bit it is roughly half a byte plus the per-group scale and zero-point metadata, which typically lands somewhere between 15 and 18 GB depending on group size.

Note that the figure that matters here is the total parameter count, not the active one. A mixture-of-experts model activates a fraction of its weights per token but stores all of them, which is why active and total parameter counts diverge so sharply for storage planning. GLM 5.2 activates around 40B of roughly 744B total, and it is the 744B that hits your disk.

Frontier open weights are a different order of magnitude

The published checkpoints for the largest open-weight models are large enough to change your hardware plan rather than your shopping list.

Kimi K3 is 2.8T total parameters and its full weights come to approximately 1.6 TB. Run the division and you get well under one byte per parameter, which tells you something useful without anyone announcing it: the release is not a 16-bit checkpoint. At two bytes per parameter that model would be over 5 TB, and the gap between those numbers is the difference between a normal server and a storage project.

This is worth doing for any model you are considering. Take the published parameter count, take the published download size, divide, and you know the native precision of the release. That in turn tells you how much headroom you have to quantise further — a checkpoint already released at low precision has far less. What a parameter count actually measures covers the input to that division.

Quantisations multiply, they do not replace

The naive plan holds one copy of each model at the precision you serve. Nobody ends up there.

In practice you keep the original release because it is the only thing you can requantise from, plus the format you serve, plus whatever you were evaluating against when you chose that format. Three copies of a large model is the normal steady state, and each one is a different size, so you cannot budget by multiplying the serving size by three.

Budget instead by summing the actual formats you intend to hold. And be deliberate about which ones earn their place: a quantisation you evaluated once and rejected is dead weight, while the original release is worth keeping precisely because re-downloading it is the slow part. Quantisation explained covers what the formats do differently.

Version sprawl grows faster than model count

The larger driver is not formats but revisions. Model families ship point releases, and the correct operational practice is to pin a specific version rather than track a moving tag, which means holding the pinned one and the candidate you are testing against it.

That is two versions per model per format, and it compounds every time a family refreshes. It is also non-negotiable if you want to roll back, because a rollback that requires a two-hour download is not a rollback. Pinning model versions covers why the discipline matters more than the disk it costs.

The practical policy is a retention rule rather than a cleanup habit: keep the current production version, the previous one, and any candidate under evaluation. Delete everything else on a schedule. Applied consistently that caps sprawl at three versions per format regardless of how often the upstream ships.

Bandwidth and cold start are the real pain

A terabyte-scale checkpoint is not expensive to store and is genuinely expensive to move. At a sustained gigabit you are looking at hours; even at ten gigabits a 1.6 TB download is not something you want in the critical path of an incident.

This changes the architecture rather than the budget. Pull once into local object storage or an internal mirror, then fan out to nodes over your own network. Never let production nodes fetch from a public hub, because you are then dependent on someone else's egress during exactly the moment you need to scale.

Loading from disk into accelerator memory is the second half of cold start, and it is bounded by disk read speed and the number of shards. NVMe with parallel shard loading turns a several-minute start into well under a minute for mid-size models. If you autoscale, that difference is the whole user-visible latency story, and it is the one part of the storage layout that end users can feel.

Storage is almost never the binding constraint

Here is the part worth internalising before you spend a week on a storage design. Disk is cheap, plentiful and easy to add. Accelerator memory is none of those things.

A model has to fit into HBM alongside the KV cache for every concurrent request, and the KV cache is what actually limits your concurrency at long context. You will hit an out-of-memory wall on the GPUs long before you run out of disk, and the model you can serve is decided there. GPU memory for inference covers that ceiling.

So size storage generously and quickly, then spend your planning effort on the memory side. A team that buys twice the disk it needs has wasted a small amount of money. A team that buys accelerators that cannot hold the model plus its cache has bought hardware it cannot use.

A sizing rule you can apply today

Take each model you intend to serve. Multiply total parameters by bytes per parameter for the format you will serve, then multiply by three to cover the original release, the serving format, and one evaluation candidate. Sum across models, add fifty percent headroom, and round up to the next sensible volume size.

Then set the retention rule before the first download rather than after the disk fills, and put the mirror in place on day one so that scaling out never touches the public internet. Both are ten-minute decisions that are considerably more annoying to retrofit.

Finally, check the storage number against the accelerator budget. If the storage plan is the expensive half of your self-hosting proposal, the proposal is probably wrong somewhere — the self-hosting break-even is set by GPUs and utilisation, not by disk.