Agentic Coding Models Compared: Who Survives the Loop

A model that writes an excellent function when asked will not necessarily survive being driven in a loop for forty turns with a shell, a file system and a failing test suite. These are different skills, and until recently the benchmarks did not separate them.

Agentic coding is the harder one. The model has to decide what to do next, call a tool correctly, read the result, notice when the result contradicts its plan, and change course without losing the thread. Every one of those is a place to fail, and failures compound rather than average out.

Why chat scores mislead here

A single-turn benchmark measures whether the model can produce a correct answer given a well-specified question. An agentic benchmark measures whether it can produce a correct answer given a vague goal, a real repository and no guarantee that any individual step works.

The compounding is the crux. If each step is 95 percent reliable, a twenty-step task succeeds about a third of the time. Small per-step differences that look like rounding error on a chat leaderboard turn into large end-to-end differences in a loop.

This is why teams keep reporting that the model with the better headline score feels worse in their agent. They are measuring different things. The anatomy of an agent loop lays out where the steps actually are.

The benchmarks that measure the right thing

Three families are worth reading in 2026. Terminal-Bench evaluates a model driving a real shell against real tasks, which exercises tool calling, error reading and recovery in one shot. SWE-bench and its harder Pro variant evaluate resolving genuine repository issues. Arena-style code rankings capture human preference on generated interfaces.

Kimi K3 leads the group on the agentic measures: 88.3 on Terminal-Bench 2.1, 81.2 on FrontierSWE, and first place on Arena Frontend Code with a score of 1,679. GLM-5.2 follows on the terminal side at 81.0, with 62.1 percent on SWE-bench Pro.

DeepSeek V4 Pro reports 80.6 percent on SWE-bench Verified, and MiniMax M3 reports a vendor-claimed 59.0 percent on SWE-bench Pro. Note carefully that Verified and Pro are different benchmarks with different difficulty — comparing a Verified number against a Pro number tells you nothing. What SWE-bench actually measures covers the distinction.

Tool calling is the load-bearing capability

Underneath all of these scores sits one skill: emitting a correctly formed tool call, with the right arguments, at the right moment. A model that reasons brilliantly and formats its function calls inconsistently is useless in an agent, because the loop breaks before the reasoning matters.

The failure modes are specific and worth testing for directly. Hallucinated parameters that do not exist in the schema. Correct schema, wrong argument values. Calling a tool when it should have answered, or answering when it should have called. Emitting several calls in parallel when your harness expects one.

None of these show up cleanly in an aggregate score, and all of them are trivially testable against your own schemas. Choosing a model for tool calling covers how to test each of them against your own schemas before you commit.

Recovery separates the field more than raw capability

The most consistent difference between models in real agent use is not how well they do the task. It is what they do when a step fails.

Strong agentic models read the error, form a hypothesis about the cause, and change approach. Weaker ones retry the identical failing command with cosmetic variations, or worse, quietly decide the failure did not happen and continue building on a broken state. The second behaviour is far more expensive, because you pay for every subsequent turn in a run that was already doomed.

Test this deliberately. Give the model a task where a tool will fail — a missing file, a permissions error, a command that does not exist — and watch the next three turns. It is a five-minute test that predicts more about production behaviour than any leaderboard. Error recovery patterns covers what good handling looks like.

Cost changes the ranking

Agentic workloads are token-hungry in a way chat is not. Every turn resends the accumulated history, so a forty-turn run does not cost forty times a single turn — it costs closer to the sum of a growing series.

That makes the price per million tokens matter more here than anywhere else. Kimi K3 lists at $3 in and $15 out per million, with cached input at $0.30. GLM-5.2 sits around $1.40 and $4.40. DeepSeek V4 Pro is roughly $0.44 and $0.87, and DeepSeek V4 Flash around $0.14 and $0.28.

The cached-input rate deserves particular attention for agents, because the resent prefix is exactly what caching is good at. An agent that reuses a stable system prompt and a stable early history can pay the cached rate on most of its input. The arithmetic of prompt caching works through the size of that effect.

How to choose

Route rather than standardise. Use the strongest model for planning and for the turns where a mistake is expensive, and a cheap one for the mechanical turns — reading a file, running a formatter, summarising output. Most agent runs are mostly mechanical turns.

If you want a single model, weigh Terminal-Bench and the agentic SWE variants above chat scores, then run the failure-recovery test on your own harness before committing. And check why benchmarks disagree before treating any two published figures as directly comparable.