Model Deprecation and Migration: Planning for the Sunset

Pinning a model version is the right call, and it comes with a bill. Pinned versions get retired, and a team that pinned eighteen months ago and never looked again is not doing an upgrade — it is doing an emergency migration on someone else's schedule.

Deprecation is the most predictable operational event in this space. It is announced in advance, the date is published, and teams still get caught. The reason is almost always that nobody owned the calendar.

How deprecation actually arrives

A provider publishes a notice: this model identifier stops serving on a date, use this replacement. The window is typically months rather than weeks, and the notice appears on a documentation page, in a changelog, and sometimes in an email to the account owner.

None of those reach the engineer who tuned the prompt. The account email goes to finance. The changelog is not in anyone's reading list. The documentation page changed silently. So the notice is real, published, and invisible.

Then the model starts returning errors, or worse, silently routes to a successor. The second case is the dangerous one, because it produces a quality change with no incident to attach it to. Model drift in production covers detecting that class of failure.

Build the inventory first

You cannot plan a migration you cannot scope. The first artefact is a list of every place a model identifier appears: application code, agent configurations, evaluation scripts, batch jobs, notebooks people actually run, and anything a contractor set up two quarters ago.

Model names are string literals, which makes them easy to grep and easy to scatter. A repository-wide search for the provider's naming pattern usually finds more call sites than anyone expected, and the surprising ones are the batch jobs.

Centralise them once you have found them. A single configuration module that maps a role — reviewer, summariser, agent — to a concrete model identifier turns a future migration from a code change into a config change. This is the single highest-leverage piece of preparation available.

Record alongside each entry which prompts are tuned against that model and which evaluation set covers it. That mapping is what tells you the true size of the work later. Pinning model versions explains why the tuning is model-specific in the first place.

Sizing the work honestly

Migration cost is not proportional to the number of call sites. It is proportional to how tightly your prompts are fitted to the outgoing model.

A simple summarisation prompt transfers to almost anything. A six-instruction prompt with negative examples, a specific ordering that suppresses one failure mode, and a fragile JSON contract is fitted to one model's quirks, and those quirks are exactly what changes. Expect to re-tune it.

Tool-calling and structured output deserve their own line in the estimate. Models differ in how strictly they honour a schema, how they behave when arguments are ambiguous, and how often they emit prose around JSON. Structured outputs and JSON mode and choosing for tool calling both matter here.

Choosing the replacement

The provider's suggested successor is a default, not a decision. A deprecation is one of the few natural moments to reconsider the whole choice, because you are paying the switching cost regardless.

This is the point to check whether the field moved underneath you. Context windows have expanded — Kimi K3, GLM-5.2, both DeepSeek V4 variants and MiniMax M3 all ship 1M — which may let you delete a retrieval layer you built around a smaller window. RAG versus long context covers when that swap is worth making.

Licensing may also have changed shape. GLM-5.2 and both DeepSeek V4 variants are MIT. Kimi K3 ships under Moonshot's own terms with a revenue threshold for model-as-a-service. If the original choice was made when your options were narrower, re-run it.

Running the cutover

Do not flip a switch. Run the new model against your evaluation set first and compare on the metrics you actually care about, not aggregate benchmarks. Parse success rate, tool-call validity, output length distribution and refusal rate catch most regressions before users do.

Then run both models on a fraction of live traffic and compare outcomes. A/B testing two models covers doing that without fooling yourself with sample sizes too small to say anything.

Keep the old identifier reachable until the last possible day, and keep the routing switch in configuration so a rollback is a deploy of one value rather than a revert of a branch. Model routing and fallbacks describes the plumbing.

Making it a standing process

The reason teams get caught is that deprecation tracking belongs to nobody. Assign it. One person, one calendar reminder per quarter, one pass over the provider changelogs for every model in the inventory.

Subscribe the engineering team — not just the billing contact — to provider status and changelog feeds. Most providers offer an RSS or email channel for API changes, and routing it into a channel people read costs nothing.

Finally, set a self-imposed staleness limit. If a pinned model has been in production for more than a year without an evaluation against current alternatives, schedule that evaluation whether or not a deprecation notice exists. A migration you chose is always cheaper than one you were handed.