AI vs AI Agent: What Actually Changes When You Add a Loop

People use "AI" and "AI agent" as if one is a fancier version of the other. They are not the same category of thing. A language model is a function. An agent is a program that calls that function in a loop and does something with the answers.

That distinction sounds academic until you try to build something, at which point it becomes the whole design.

A model is a pure function

Strip away the chat interface and a large language model does exactly one thing: given a sequence of tokens, predict the next one. Wrap that in a loop over its own output and you get text generation. That is the entire primitive.

messages -> [model] -> text

It has no memory between calls. It cannot read a file, run a test, or check whether what it just told you is true. Every request is independent. If you ask it what time it is, it will produce something that looks like a time, because plausible-looking output is the only thing it is optimised for.

This is not a flaw. It is what makes models composable. A function with no side effects is a thing you can build on.

An agent is a loop with hands

An agent wraps that function in three additions:

1. Tools — functions the model can ask to have run. Read a file, run a shell command, query a database, call an API.
2. A loop — the model's output feeds back in as new input, so it can react to what happened.
3. A stopping condition — some way to decide the job is done, or that it has gone wrong.

while not done:
    reply = model(messages)
    if reply.wants_tool:
        result = run_tool(reply.tool, reply.args)
        messages.append(result)      # the model now sees what happened
    else:
        done = True

That is genuinely most of it. The sophistication in a good agent is not in the loop — it is in the tool design, the error handling, and knowing when to stop.

The difference in practice

Ask a model: "Why is my test failing?" It sees your message and nothing else. It will guess, and it will guess confidently, because it has no way to look.

Ask an agent the same thing and a competent one will run the test, read the actual stack trace, open the file mentioned in the trace, and then tell you — grounded in output it fetched rather than output it invented.

Same model. Completely different reliability, because the second one could check.

Why this makes token costs unpredictable

Here is the consequence nobody mentions until the bill arrives.

A chat turn is one request. An agent turn might be fifteen — and each one resends the entire accumulated conversation, because the model is stateless. Turn 10 carries turns 1 through 9 in its input.

Input tokens therefore grow roughly quadratically over a session. A single "fix this bug" instruction that runs for twenty tool calls can consume more tokens than a hundred chat messages. This is why developers who move from chat to agentic tools are so often shocked by their first metered invoice: the work looks similar and costs an order of magnitude more.

It is also why flat-rate access and agentic coding fit together so well. When the meter is not running, you stop rationing the loop.

What agents are still bad at

Being honest about this saves you a lot of wasted effort:

• Long horizons. Reliability degrades with every step. An agent that is 95% reliable per action is about 36% reliable over twenty actions. Short, checkable tasks work; "build me a startup" does not.
• Knowing when they are stuck. Models are trained to produce plausible continuations, and "I cannot do this" is rarely the most plausible continuation. Agents loop on the same failed approach unless you make them stop.
• Irreversible actions. An agent that can delete things will eventually delete the wrong thing. Gate destructive tools behind confirmation.

Choosing between them

Use a plain model call when the task is one-shot and self-contained: summarise this, rewrite that, classify these. Adding a loop buys you nothing and costs latency.

Use an agent when the task requires finding something out — when the right next step depends on information the model does not have yet. Debugging, refactoring across files, anything involving your actual codebase.

The test is simple: does the task require looking at something? If yes, you want an agent. If no, you are just paying for extra round trips.