Should prompts live in the repo or in a database?

The repo by default: you get review, history and atomic deploys with the code. Move to runtime config only when non-engineers own the wording, and then apply feature-flag level controls.

Why log a content hash if I already have a version number?

Because someone will edit a template without bumping the version. The hash makes that visible in your logs instead of turning into an unreproducible bug report months later.

Does whitespace in a prompt template really matter?

It changes the token sequence, which can change output and will break an exact-prefix cache match. Use a templating engine with whitespace control and assert on the rendered string in a test.

Prompt Templates and Versioning: Treating Prompts as Code

A prompt is an input to a program that has no compiler, no type system and no stack trace. Change a word and the behaviour changes; nothing tells you it changed. The engineering problem is not writing the prompt, it is knowing which prompt ran, why it was edited, and whether the edit helped.

Most teams start with prompts as f-strings scattered through handlers. That survives about three months. Then someone tweaks a sentence to fix one customer complaint, ships it, and a different customer's workflow breaks in a way nobody connects to the change for a fortnight.

Templates are string concatenation with a footgun

The moment you interpolate user content into a prompt, you have an injection surface and a formatting surface. Both bite.

The formatting surface is the one people underestimate. Leading whitespace from a triple-quoted Python string, a trailing newline that appears only when a variable is empty, a list rendered with a comma when it has one element and a newline when it has several. Each of these changes the token sequence, and a changed token sequence can change the output. It also invalidates a prefix cache, which turns a cosmetic edit into a cost regression.

Use a real templating engine with explicit whitespace control rather than string addition. Jinja with trim_blocks and lstrip_blocks, Handlebars, or your language equivalent. Render the template to a string and assert on that string in a test, so that whitespace changes show up as diffs rather than as mystery behaviour shifts.

SYSTEM = env.get_template("classify/system.j2")
rendered = SYSTEM.render(labels=labels, today=today)
assert rendered == golden_text  # fails loudly on invisible edits

Separate the parts that change on different clocks

A prompt usually contains four things with completely different update frequencies: the role and task definition, the output contract, the examples, and the runtime data. Bundling them into one string means a schema change forces you to re-read the instructions, and adding an example risks editing the contract.

Split them into separate files and compose at render time. The task definition changes when the product changes. The output contract changes when the schema changes, and should be generated from the schema rather than hand-written twice. Examples change constantly and belong under their own management discipline, which is a large enough topic to have its own set of rules for curating and rotating example sets.

Runtime data should never be concatenated into the instruction block at all. Put it in a separate message, delimited, and say in the instructions that everything inside the delimiters is data. This does not make prompt injection impossible, but it removes the trivial cases and gives you somewhere to sanitise.

Version the prompt, not the file

Git already versions the file. What you need is a version identifier that travels with the request and lands in your logs, so that six weeks later you can answer "which prompt produced this output".

Give every template a stable name and a monotonic version, and compute a content hash at load time. Log the name, the version, the hash, the model ID and the rendered length on every call. The hash catches the case where someone edits a template without bumping the version, which will happen.

{"prompt": "extract/invoice", "version": 7,
 "hash": "9f2c41ab", "model": "kimi-k2.6", "tokens_in": 3184}

Pin the model alongside the prompt. A prompt and a model are one unit of behaviour: the same text against a different checkpoint is a different program. If your provider offers dated model IDs, use them, for the reasons set out in the case for pinning model versions rather than tracking a floating alias.

Roll forward, never edit in place

Treat a shipped prompt version as immutable. To change behaviour, add version 8 and route traffic to it. This costs a few kilobytes and buys you an instant rollback that does not require a deploy, plus the ability to run both versions side by side on the same traffic.

Routing by version also makes staged rollout possible. Send five percent of requests to the new version, compare outcomes on the metric you actually care about, then move the rest. Without a version identifier in the request path you cannot do any of this, which is why the identifier matters more than the storage format.

Keep the old versions in the repository even after they stop serving traffic. When a customer reports something odd from last month, you want to reproduce it exactly, and that means rendering the template as it was.

Where the prompts live

Two workable answers, and one that is not.

Prompts in the repository, deployed with the code, is the right default. You get review, history, branching, and atomic deploys with the code that depends on them. The cost is that a copy tweak needs a deploy, which annoys non-engineers.

Prompts in a database or a config service, fetched at runtime, suits teams where subject-matter experts own the wording. The cost is real: you now have two deploy pipelines, and a prompt can change without any code review. If you go this way, insist on the same controls you would apply to a feature flag, and cache aggressively so a config outage does not take down inference.

The answer that is not workable is prompts edited directly in a vendor playground and copied into production by hand. There is no history, no review and no way to reproduce an old output.

Structure the prompt so caching survives edits

Providers that support prefix caching only reuse a cached prefix if the tokens match exactly from the start. That makes ordering an engineering decision rather than a stylistic one: stable content first, volatile content last.

Put the system role, the output contract and the fixed examples at the top; put the retrieved documents, the user turn and anything containing a timestamp at the bottom. Injecting today's date into the first line of a system prompt busts the cache once a day for every request. The mechanics are worth understanding before you optimise, and the way prefix caching actually matches tokens explains what counts as a match.

A minimum viable discipline

You do not need a platform. You need six things: templates in files rather than strings, an engine with explicit whitespace control, a name and version per template, a content hash logged with every call, immutable versions with roll-forward changes, and a golden-render test so invisible edits fail in CI.

Add the seventh when the first regression bites: a small evaluation set that runs against a candidate version before it takes traffic. That is a separate discipline with its own statistics problem, covered in how to tell whether a prompt change actually improved anything, and it is the difference between versioning and guessing.

Once more than one team is writing prompts, the naming and ownership questions get their own weight, and the practical shape of that is described in building a prompt library people actually reuse.