Generating API Clients With an LLM Without Silent Drift

If you have a complete OpenAPI document, a language model is the wrong tool for the mechanical half of client generation. openapi-generator will emit the same types from the same spec every single run, in every language, for free. A model will emit something slightly different each time and occasionally invent a field.

That is not an argument against using a model here. It is an argument about which half of the problem you point it at. The mechanical half is solved. The half that is not solved is everything the spec fails to encode, and that is where most of the pain in a hand-maintained client actually lives.

Establish what the spec already gives you

Start by running the deterministic generator and reading the output honestly. For a well-specified API you will get request and response types, path construction, and query parameter serialisation that is correct and that you never have to review again.

What you will also get is a client nobody enjoys using: types named after schema components rather than concepts, every field optional because the spec never marked anything required, no retry behaviour, and error handling that surfaces a raw status code and a body of unknown shape.

Write down the delta between that output and the client you actually want. That list is your prompt. It is a much better brief than pointing a model at a spec and asking for an SDK, because the model is now solving a stated ergonomics problem instead of redoing work a deterministic tool did better.

The undocumented behaviour is the real target

Most APIs are documented in prose that the spec does not capture. The rate limit resets on a header you have to read. A 200 response with an empty body means the resource was queued, not created. A cursor is opaque and must not be parsed. One endpoint returns an array at the top level and every other one wraps it in a data object.

Feed the model the prose documentation alongside the spec and ask it specifically to produce the wrapper behaviour: the pagination iterator, the retry policy keyed to the documented headers, the discriminated union over the documented error codes. This is a translation task from natural language to code, which is what these models are genuinely good at.

Ask for the resulting behaviour to be stated as comments citing the sentence in the documentation that justified it. When a maintainer later wonders why the client sleeps for the value of a particular header, the answer is in the file rather than in someone's memory.

Slice the spec; never paste the whole thing

A large public API spec runs to tens of thousands of lines, and pasting all of it produces worse output than pasting the relevant fragment. The model spends its attention budget on endpoints you are not generating, and the accuracy on the ones you are generating drops.

Resolve the references yourself first. For a single endpoint, extract the path item, the request body schema, the response schemas, and every component schema reachable from those, and send that closed subgraph. A short script over the parsed document does this reliably, and it turns a 40,000-line input into a few hundred lines.

Generate one endpoint group per call, in a fresh context, with the shared types already written and passed in as a fixed preamble. This costs more calls and produces far more consistent output than one enormous request, for reasons the lost in the middle problem explains well: material buried in the centre of a long input is attended to less reliably than material at either end.

The compile loop is the only verification that counts

The characteristic failure of generated client code is not a syntax error. It is a plausible field name that does not exist, a nullable field treated as required, or an enum missing its least common member. All of those look correct on review.

So do not review first. Compile first, then run a validation pass that checks every generated type against the spec programmatically: does each property name appear in the schema, does each required list match, does each enum have the same member set. That comparison is a hundred lines of script and it catches the entire class of invented-field errors that human review misses.

Then feed failures back. A loop that compiles, diffs against the spec, and returns the specific mismatches to the model converges in two or three iterations. A loop that just returns the compiler output converges more slowly, because the type checker cannot see the spec and therefore cannot tell you that a field name is fictional.

Cap the iterations. If it has not converged after four attempts, the endpoint has something unusual in it and a person should look. Unbounded repair loops are a known way to spend a lot of money making no progress, which is covered in more depth in detecting agent loops.

Pagination, retries and errors deserve hand-written tests

These three are where deterministic generators are weakest and where a model is most useful, which unfortunately also means they are the parts with no reference implementation to diff against.

Write the tests before generating. A fake server that returns three pages and then an empty cursor, one that returns 429 with a reset header, one that returns a malformed body. The generated client either drives those correctly or it does not, and you find out in seconds instead of in production. The taxonomy in the LLM API error codes reference is a reasonable template for the error union even when the API you are wrapping is not an inference API.

Be particularly suspicious of generated retry code. Models default to retrying everything, including non-idempotent POSTs and 400-class errors, and they frequently retry without jitter. Specify the policy in the prompt rather than accepting whatever comes back, and check it against how rate limits and retries should actually behave.

Regeneration is a diff problem, not a generation problem

The client is generated once and maintained for years. The question that decides whether this approach survives is what happens when the API adds a field.

Keep generated code in files that are never edited by hand, and put every hand-written adjustment in a separate layer that imports from them. The moment someone patches a generated file, regeneration becomes a merge conflict and the team stops regenerating, and the client drifts from the API silently.

Commit the spec alongside the generated output. Then regeneration is a diff of two spec versions plus a diff of two generated outputs, and a reviewer can check that the second follows from the first. Pin the model version too, for the same reason you would pin a model version anywhere else — an unannounced upgrade that restyles the whole client turns a two-field diff into an unreviewable one.

When to skip the model entirely

If the API has a maintained official SDK in your language, use it. If the spec is complete and accurate and you only need types, run the deterministic generator and stop. If the API has four endpoints, write the client by hand in twenty minutes.

The case where this pays is a mid-sized API with real prose documentation, no official SDK for your language, and behaviour the spec does not encode. That is common, and the saving is measured in days.

The decision rule: use a deterministic generator for anything the spec fully describes, use a model for the ergonomics and behaviour the spec omits, and verify both against the spec in code rather than by reading. If you cannot automatically prove the generated types match the schema, you are not generating a client — you are writing one slowly with extra steps.