PII in Prompts: How It Gets There and How to Get It Out

Almost nobody sets out to send personal data to a model provider. The prompt template asks for a code snippet, a log excerpt, a ticket summary. The personal data arrives anyway, embedded in the thing that was legitimately being sent, and it arrives through a path nobody reviewed because the review was of the template rather than of what fills it.

That is the useful framing. PII in prompts is rarely a design decision you can argue about; it is a leakage path you have to instrument. The work is finding the paths, deciding what happens at each boundary, and knowing what your provider does with the bytes after they arrive.

The paths it actually takes

Stack traces are the most common. An exception message frequently carries the value that caused it — the malformed email address, the customer name that broke a parser, a serialised request object with an address in it. Paste the trace into a debugging prompt and the value goes with it.

Test fixtures are the second. A team seeds a staging database from a production dump, generates fixtures from staging, and now the repository contains real names and real order histories in files that look synthetic. Any agent with read access to the repo will send them without anything unusual happening.

Support tickets and CRM records are the third and most direct: triage and summarisation features exist to consume text customers wrote about themselves. Log analysis is the fourth, since request logs carry IP addresses, session identifiers and full URLs with parameters. Using an LLM for log analysis covers that workload.

Detect at the boundary you control

The only reliable place to check is immediately before the outbound call, in the client or proxy that every request passes through. Checking earlier means each service reimplements the policy and one of them gets it wrong. Checking later means it already left.

Detection is a mix of two approaches and both are needed. Pattern matching handles the structured identifiers — email addresses, phone numbers in the formats you actually see, national insurance and social security numbers, card numbers with a Luhn check to cut the false positives, IP addresses. It is cheap, deterministic, and catches most of the volume.

Names, addresses and free-text disclosures need classification rather than patterns, which is harder and less accurate. Be honest about the residual rate rather than describing a regex pass as coverage: the realistic goal is catching the structured majority reliably and reducing the rest, not reaching zero.

Substitute placeholders and rehydrate on the way back

Blanket redaction breaks the task. If you replace every name with a fixed marker, a model asked to draft a reply to a customer produces something you cannot send, and a model asked to reason about which of two users made a change loses the distinction entirely.

Consistent tokenisation solves both. Replace each distinct value with a stable placeholder for the duration of the request — the first person becomes one token, the second another — and keep the mapping in memory on your side. The model sees structure and relationships without seeing identities, and its output comes back containing the same placeholders.

Substituting the real values back in on the response path gives you a usable answer with the personal data never having left your process. Keep the mapping request-scoped: persisting it recreates the identifier you were trying to avoid sending. Rehydration should also fail loudly on a placeholder that was never issued, because models occasionally invent one.

Read the retention terms, not the marketing page

Providers differ substantially on what happens to a request after it is served, and the difference is contractual rather than technical. The questions worth answering before a workload carrying personal data is pointed at an endpoint are specific.

How long are requests and responses retained, and is that window different for abuse monitoring than for the API itself? Is there a zero-retention or no-logging mode, is it available on your plan, and is it per-request or account-wide? Are inputs used for training, is opting out the default, and does the default differ between the consumer product and the API?

Then the operational half: which subprocessors see the data, in which jurisdictions is it processed, is there a data processing agreement covering your regulatory position, and what is the breach notification commitment. The provider selection checklist covers the wider version of this conversation.

Where self-hosting genuinely changes the answer

Running open weights on infrastructure you control removes the third party from the data path entirely, which for some regulatory positions is the difference between a workload being possible and not.

It does not remove the problem. The prompts still exist, they still get logged, and a self-hosted deployment with default logging turned on can retain more personal data for longer than a managed provider with a zero-retention mode. The control moves to you; the obligation does not disappear.

Treat it as a decision about which risk you would rather manage rather than an elimination of risk. Local models versus an API covers the trade-off, and the cost comparison covers what it costs to hold that control.

Make the boundary auditable

Whatever you build, you will eventually be asked to describe it precisely — by a customer questionnaire, an auditor, or an incident review. Vague answers are expensive at that point.

Record what the detector found rather than what it found it in. A count of matches by category, the placeholder count, and whether the request was blocked or transformed gives you evidence the control ran without creating a second copy of the data you were protecting. Logging model requests safely covers keeping that record minimal.

The practical sequence is: enumerate the paths in your own system, put detection at the single outbound boundary, tokenise rather than redact so the task still works, confirm retention terms in writing before the first request, and log that the control fired rather than what it caught. Keeping credentials out of prompts is the same boundary doing adjacent work.