Mistral Medium 3.1: Evaluating a Point Release With Few Numbers

Mistral Medium 3.1 is a useful case study in a problem every team meets: a vendor ships a new version, the release note is one line long, and you have to decide whether to switch. Mistral's model card describes it as its "frontier-class multimodal model released August 2025. Improving tone and performance." That is close to the whole of the official account.

This guide covers what is known about 3.1, how to evaluate a point release when the vendor publishes few numbers, and where 3.1 stands now that Mistral has deprecated it.

What 3.1 is, and what was published

Mistral's documentation dates the release of Medium 3.1, as mistral-medium-2508, to 12 August 2025, three months after Medium 3. The model card lists a 128K context window and a familiar feature set: structured outputs, function calling, predicted outputs, document question answering, prefix completion, batching, agents and conversations, and built-in tools.

What it does not list is a benchmark table comparing it with Medium 3. Like its predecessor, 3.1 is listed as a Premier model rather than an open-weight release, so there is no public checkpoint to test independently either.

The price did not change. OpenRouter lists it at $0.40 per million input tokens and $2 per million output at the time of writing, identical to Medium 3, and the window is the same. On paper, it is a drop-in replacement.

Why "improved tone" is worth testing

It is tempting to dismiss a tone change as cosmetic. For a user-facing product it is not. Tone covers verbosity, hedging, formatting habits, how often the model adds unrequested caveats, and whether it answers the question or restates it first. Those properties decide whether a support assistant reads as helpful or as padded, and whether downstream parsing that expects a certain format keeps working.

A tone change also cuts both ways. A model tuned to sound warmer may produce longer answers, which costs more output tokens. A model tuned to be more concise may drop detail your users relied on. Neither shows up on a reasoning benchmark.

How to test a point release

When the vendor gives you a sentence instead of a table, build the table yourself. The method is the same for any minor version bump:

1. Freeze a regression set. Take fifty real requests from your logs, covering your common cases and the ones that have broken before. Store the inputs and the outputs you accepted.
2. Run both versions side by side. Same prompts, same temperature, same tools. Use dated ids — mistral-medium-2505 and mistral-medium-2508 — so you know exactly what you compared.
3. Measure the boring things. Output token count per request, schema validity, tool-call correctness and refusal rate. These shift more often than headline quality does.
4. Read a sample blind. Shuffle twenty pairs, hide the labels, and pick the better answer. Tone differences are obvious to a human reader and invisible to most automated scores.

How to benchmark LLMs on your own work covers the harness in more depth. The point is that a release note is a hypothesis, and your own traffic is the only test that settles it.

The batch endpoint

OpenRouter also lists mistral-medium-3.1:batch. It is the same model behind an asynchronous endpoint: you submit requests in bulk, results arrive later, and the listed price halves to $0.20 input and $1.00 output per million tokens at the time of writing. Mistral's model card lists batching as a supported feature on its own API too.

For offline work such as classifying a backlog, generating descriptions or running evaluations, the batch route is the obvious choice. It is also a cheap way to run the regression comparison above.

Where 3.1 stands now

Mistral's documentation marks Medium 3.1 as deprecated from 22 May 2026, with retirement scheduled for 31 August 2026 — the same dates as Medium 3 — and names Mistral Medium 3.5 as the replacement for new integrations. That retirement date has now passed on Mistral's own schedule. Third-party listings can outlive the vendor's dates, but building on a deprecated model means accepting a failure you cannot schedule.

Moving to 3.5 is a bigger step than moving from 3 to 3.1 was:

• Price rises from $0.40/$2.00 to $1.50/$7.50 per million tokens on Mistral's API.
• Context doubles from 128K to 256K.
• Reasoning becomes a per-request option through reasoning_effort, which adds output tokens when enabled.
• Weights become available under a modified MIT licence, with a revenue threshold for large companies.

That is not a point release, and it deserves the full evaluation above rather than a quick swap. The Mistral Medium 3.5 guide covers the new model, and the Mistral Medium 3 guide covers the original release that 3.1 refined.

The takeaway

Medium 3.1 was a reasonable upgrade for anyone on Medium 3: same price, same window, same features, better tone by Mistral's account. Its lasting value is as a reminder of process. When a vendor ships a version with a one-line note, keep a regression set ready, compare on dated ids, and measure output length alongside quality. You will need the same set again when the next version arrives, and with 3.1's retirement, that moment has already come.