How to Read a Model Card Without Being Misled

A model card is the closest thing to a datasheet the field has. It is also written by the vendor, structured however they like, and free to be silent about anything inconvenient.

Reading one well is mostly about knowing which fields are load-bearing facts, which are marketing framed as measurement, and what an absent section is telling you.

The fields that are simply true

Architecture and size are reliable. Total parameters, active parameters per token, expert count, attention variant — these are structural facts the vendor has no reason to misstate, and open-weight releases can be checked against the files.

Kimi K2.6 describing itself as 1T total with 32B active across 384 experts, 8 routed plus 1 shared, with MLA attention, is exactly the kind of statement to take at face value. It also tells you a great deal: the routed-expert design is why a trillion-parameter model can serve at a moderate price.

Context length is factual but needs a qualifier. A card advertising 1M is telling you what the API accepts, not what the model uses well, and recall in the middle of a long window is a separate question the card will not answer.

Maximum output length is worth finding and is frequently buried. GLM-5.2 listing 128K max output alongside its 1M input is the sort of detail that decides whether a long-document rewrite is one call or twelve.

Licence text deserves more attention than it gets

This is where the most expensive surprises live, and where "open" does the most work as a word. DeepSeek V4 Pro and Flash are MIT, and GLM-5.2 is MIT, which means you can do essentially anything including reselling access.

Custom licences are a different proposition. Kimi K3 ships open weights under Moonshot's own terms, with a revenue threshold that attaches conditions once you offer the model as a service at scale. That is entirely workable, and it is not MIT, and the distinction matters the moment your product is the model.

Read for the specific clauses rather than the headline: acceptable use restrictions, attribution requirements, whether outputs may train other models, and whether the licence can change on a future release. Open weights and open source are not the same thing, and cards routinely blur them.

If your plan involves redistribution or a hosted offering, get the licence read by someone whose job that is before the model gets embedded in a roadmap.

The evaluation section, read sceptically

Every benchmark table on a card is self-reported. That does not make it wrong, but it does make it a ceiling produced by a tuned harness — the distinction covered in vendor-claimed versus independent benchmarks.

Check the variant names before anything else. A card reporting SWE-bench Verified and a competitor reporting SWE-bench Pro are not in a race, and cards do not always make the variant prominent.

Look for the comparison set the vendor chose. Which models appear in the table, and more tellingly which do not, is a decision made after the results were known. An absent obvious competitor is information.

Finally, note which benchmarks are missing entirely. A card heavy on bounded coding scores with nothing on long-horizon agentic work is not necessarily hiding a weakness, but it is the first thing to test yourself.

What omission tells you

Cards are rarely false and often incomplete, so read the gaps. No training data description means you cannot reason about domain coverage or contamination risk. No tokenizer detail means you cannot predict cost on non-English text, which varies more than people expect.

No latency or throughput numbers is normal but worth noting, because serving characteristics are often the operational constraint rather than quality. No safety or refusal discussion means you will discover the boundaries in production.

For open-weight releases, check whether the card states the actual on-disk size. Kimi K3 weights at roughly 1.6TB is the number that determines whether self-hosting is a conversation at all, and it is easy to skim past.

Where the card is silent and the answer matters, the fix is to measure rather than to email support. One afternoon with your own tasks resolves more than a week of documentation archaeology.

A ten-minute reading order

Start with the licence, because a licence problem invalidates everything else. Then the hard facts: total and active parameters, context in and out, and price per million tokens in and out with the cached input rate noted separately.

Next, the shape. Dense or sparse, single modality or multimodal, whether reasoning effort is a knob you control. These determine how the model behaves under your workload more than any benchmark number will.

Only then the evaluation table, read as a ceiling, with the variants checked. Note which benchmarks are absent and add those to your own test plan.

Close by writing down the two or three claims your decision actually depends on, and test exactly those. Evaluating models on your own codebase turns a card from a sales document into a hypothesis you can check.