ModelsMistral Small 3 (24B, 2501): The Latency-First Open Model
Mistral Small 3 is a 24B Apache 2.0 model trained without RL or synthetic data, tuned for speed over depth. Its specs, its 32K limit, and when it is still worth running.
Most model releases chase a higher score. Mistral Small 3, published on 30 January 2025, was explicitly designed around a different target: answer quickly, on hardware a single team can own, with a model that is easy to build on. The weights carry the identifier Mistral-Small-24B-Instruct-2501, where 2501 is the year and month of release.
It has since been replaced twice in its own family and retired from Mistral's API. But the design decisions behind it explain the whole Small line, and they still make it a sensible base for some jobs.
The design brief: fewer layers, faster answers
Small 3 is a dense 24B model. The launch post gives no layer count, only that it uses "far fewer layers than competing models", and credits that choice for its speed. The launch post claims around 150 tokens per second and says the instruct model is more than three times faster than Llama 3.3 70B on the same hardware while staying competitive with larger models.
The use cases Mistral listed follow from that: fast conversational assistance, low-latency function calling in agent workflows, and local inference for sensitive data. Once quantised, Mistral says it fits on a single RTX 4090 or a MacBook with 32 GB of RAM. Unquantised, the model card puts serving at about 55 GB of GPU memory in BF16.
If you are choosing a model for interactive work, time to first token and output speed often matter more than a few benchmark points, and they depend on the host and hardware as much as on the model. Measure them on the endpoint you will actually use.
No RL, no synthetic data: why that matters for fine-tuning
The detail that most distinguishes Small 3 is a sentence in the announcement: it is "neither trained with RL nor synthetic data". Mistral positioned that as making it a good base for further training, and released a pretrained base checkpoint alongside the instruct model.
The reasoning is straightforward. A model that has already been through heavy preference optimisation carries strong habits — refusal patterns, a house style, verbosity — that a downstream fine-tune has to fight. A plainer starting point is easier to steer toward a domain voice such as legal, medical or support work, which are exactly the examples Mistral gave. Fine-tuning vs prompting vs RAG covers whether you should be fine-tuning at all.
The weights are Apache 2.0, so a tuned derivative can be shipped commercially without a separate agreement.
What the benchmarks say, and who ran them
The model card reports these figures for the instruct model:
- MMLU-Pro — 66.3
- GPQA Main — 45.3
- HumanEval (pass@1) — 84.8
- MATH — 70.6
- IFEval — 82.9
- Arena Hard — 87.3, MT-Bench — 8.35, WildBench — 52.27
The launch post separately claims over 81% on MMLU. The last three numbers are judge-based, scored by GPT-4o (2024-05-13) acting as grader, and Mistral notes its results may differ slightly from other published figures. Judge-based scores reward a style the judge likes as much as correctness, so treat them as a rough signal of chat quality rather than a measure of accuracy. And all of these were run by the vendor, on its own harness, against 2025 rivals.
The 32K window is the real constraint
The model card gives a 32K context window, and that is the number most likely to rule Small 3 out today. Long documents, multi-file code context and long agent transcripts will not fit, and you will be doing retrieval or compaction work that a larger window would avoid.
It is also text only. Image input arrived with Mistral Small 3.1 in March 2025, which kept the same 24B size and raised the window to 128K. If you need either, that is the reason to move up the family, and Mistral Small 3.2 is the more polished version of that same upgrade.
Availability in 2026
Mistral's documentation lists Small 3.0 as deprecated on 6 November 2025 and retired on 30 November 2025, with Mistral Small 4 as the suggested replacement. Third-party hosts still serve the open weights; at the time of writing it is listed on OpenRouter at $0.05 per million input tokens and $0.08 per million output, with JSON mode and structured outputs listed as supported. Check current rates before budgeting.
The model card describes native function calling when served through vLLM with --tool-call-parser mistral and --enable-auto-tool-choice, and recommends a low temperature of 0.15. Whether a given hosted endpoint exposes tool calling is a property of that host, not the model, so check before you wire an agent to it.
When to still pick it
- Pick Small 3 if you need a clean, permissively licensed 24B base for your own fine-tune and your inputs fit in 32K.
- Pick Small 3.2 if you want the same size with vision, a longer window and better instruction following.
- Pick Mistral Small 4 for a new hosted integration. It is the only member of the family still on Mistral's API, though it is a much larger model to self-host.
Whatever you choose, run the candidates on twenty of your own prompts before committing. A latency-first model earns its place by being fast enough on your workload, and only a test on that workload tells you whether it is.
Common questions
What is the context window of Mistral Small 3?
The model card for Mistral-Small-24B-Instruct-2501 gives 32K tokens. Mistral Small 3.1 and later raised it, so if your prompts exceed 32K move up the family rather than fighting the limit.
Can Mistral Small 3 run on a single consumer GPU?
Quantised, Mistral says it fits on one RTX 4090 or a MacBook with 32 GB of RAM. Unquantised in BF16 the model card puts serving at about 55 GB of GPU memory.
Why does it matter that Small 3 was not trained with RL?
A model without heavy reinforcement learning or synthetic data carries fewer ingrained habits, which makes it easier to fine-tune toward a specific domain or voice. Mistral released a base checkpoint for that purpose.
