GDPR and HIPAA in LLM prompts: redact, rehydrate, keep nothing

What an auditor actually looks at when an LLM feature touches regulated data, and how the redact and rehydrate pattern maps to data minimization, retention, and DPA obligations.

This isn't legal advice. Talk to your DPO or counsel before making compliance decisions based on a blog post.

The product team wants to ship a chat assistant for nurses. The model is going to read patient notes, the user is going to ask follow-up questions, and the answer is going to come back into a clinical workflow. Engineering says it's a two-week build. Then legal sits down at the meeting and asks three questions: where does the prompt go, what is in it, and how long does the vendor keep it.

Every one of those questions is about the same thing: the boundary between your service and the LLM provider. That boundary is where compliance lives. Most of the work of making a regulated LLM feature shippable is making that boundary defensible, and a lot of that work looks suspiciously like good redaction.

We build peyeeye. We're biased about what good redaction looks like, but the compliance vocabulary is older than any of us, so let's walk through it on its own terms first.

GDPR LLM compliance vocabulary, mapped to redaction

Compliance reviewers don't care about your tokenizer. They care about a small set of legal principles, and they want to see how your design honors each one. Here are the ones that come up every time, and the way the redact-and-rehydrate pattern maps to them.

Data minimization at the prompt boundary

GDPR Article 5(1)(c) says personal data shall be “adequate, relevant and limited to what is necessary” for the purpose. HIPAA calls the same idea the “minimum necessary” standard. Both rules are asking the same question: did you send more than you needed to.

For an LLM call, the answer is almost always yes if you didn't redact. The model needs the shape of the question, not the patient's name. It needs the structure of the address, not the actual street. Replacing identifiers with placeholders before the prompt leaves your service is data minimization at the prompt boundary, expressed in code. It's not the only thing you have to do, but it's the thing that's easiest to demonstrate in an audit. The diff between “send the raw note” and “send the note with names, MRNs, and addresses tokenized” is a one-liner that a reviewer can read.

Purpose limitation: the model generates, it doesn't store

Article 5(1)(b) says data collected for one purpose can't be quietly reused for another. The LLM's purpose in your pipeline is generation. It's not a database, not an analytics warehouse, not a future training corpus. The trouble is that you have to take the vendor's word for that, and the vendor's word lives in their terms of service.

If the prompt only contains placeholders, the question of what the vendor does with it gets a lot smaller. A request that reads “summarize the visit for [PATIENT_1] on [DATE_1]” isn't personal data in the way the raw version is. The placeholders can't be re-identified without the mapping, and the mapping never went to the model vendor. That's a defensible purpose-limitation story.

Storage limitation and zero retention

Article 5(1)(e) says you keep personal data only as long as you need it. Auditors love this one because it's falsifiable: either there's a retention timer, or there isn't. With most redaction setups, the token map is the thing with the retention problem. The map is sensitive (it links a placeholder back to a real value), and if your service holds it forever, you've quietly built a parallel database of identifiers.

peyeeye has a keep-nothing mode, and it's the one to reach for here. The default stateful mode holds the token map server-side for the life of the session so that rehydrate works across calls; that map is a data store with a retention question, and it belongs in your DPIA like any other. Stateless mode is the zero-retention path: the mapping is sealed into an AES-GCM blob (the skey_… token), handed back to your service, and never stored on our side at all. In that mode, our database has no idea what your placeholders refer to. We've written more about the trade-offs in a follow-up post on stateless versus stateful sessions; for this one, the relevant bit is that “keep nothing” is a configuration choice, not a quarterly cleanup job.

DPA and BAA: the contracts you still need

Redaction reduces the surface, but it doesn't replace contracts. You still need a Data Processing Agreement with anyone who processes personal data on your behalf, and for HIPAA-regulated work in the US you still need a Business Associate Agreement with anyone who touches PHI. That includes the LLM vendor and, in most cases, the redaction vendor.

For peyeeye specifically: we're designed for regulated environments, and stateless mode gives you zero retention by construction, but a BAA-covered deployment for PHI is an enterprise conversation, not a checkbox on the signup page. If you're building something where PHI hits the wire, talk to us before going to production. The same is true of any third-party tool you bring into the path. A clean technical pattern with no paperwork is still a finding.

Things to ask any vendor in the LLM pipeline:

• Will you sign a DPA, and on whose template.
• For US healthcare, will you sign a BAA. If yes, on which plan.
• Where does the data sit at rest, and where does it transit.
• What's the default retention, and can it be set to zero.
• Do you train on customer data, by default or ever, and how do we opt out.
• What's the sub-processor list, and how are changes notified.

Right to erasure and the “keep nothing” shortcut

GDPR Article 17 gives data subjects the right to have their data deleted. The cleanest way to honor that right is to not have stored the data in the first place. If your redaction layer never persists the mapping, there's nothing to erase. If your LLM provider doesn't retain the prompt, there's nothing to erase there either. A stateless flow turns Article 17 from a quarterly engineering project into a one-line response: we don't store it.

That's not a magic trick, it's a design choice. You give up some convenience. You can't look back at last week's redacted prompts and ask “who was [PATIENT_1] in this transcript.” If your product needs that, you'll want stateful sessions, and your retention policy goes back to being something you have to write down and enforce.

The honest part: what redaction does not get you out of

We've seen redaction pitched as a compliance silver bullet. It isn't, and treating it as one will cost you when an auditor asks the second question.

Things redaction does not solve:

• Lawful basis for the underlying processing. If you don't have consent, contract, or another lawful basis to process the patient's record at all, redacting it before sending it to the model doesn't fix that. The processing already happened on your side.
• The BAA requirement for PHI. Redaction reduces what crosses the boundary, but the standard reading of HIPAA is that you still want a BAA with anyone in the data flow that's reasonably likely to touch PHI. A redaction model that sometimes misses a name has touched PHI, even briefly.
• Detection accuracy. No detector is perfect. Free-text clinical notes will eventually contain a name in a format your regex didn't see. Layering an ML backend on top of regex helps; assuming any of it is 100% does not.
• Output safety. The model can hallucinate identifiers that look real. That's a separate problem from input redaction, and it needs its own controls.

Redaction is a load-bearing piece of the compliance story, but it's one piece. We like to say it makes the boundary defensible; we don't say it makes the whole system compliant.

The four-step pattern, viewed from compliance

The shape of a HIPAA LLM prompt path is the same as any other LLM prompt path: detect, redact, prompt, rehydrate. We've covered the engineering version of this in how to redact PII before an LLM call. Here's how each step reads to a reviewer.

1. Detect. Run a regex pass with checksums (Luhn, mod-97, SSN structure) and an optional ML pass for free-text names and addresses. The reviewer wants to see that you have both, and that you can show what entity types are covered.
2. Redact. Replace each detected span with a deterministic placeholder. Keep the mapping either in a session you control, or in an opaque sealed blob you don't.
3. Prompt. Send the redacted text to the LLM. The reviewer wants to see that the prompt logged on your side and the prompt logged on the vendor's side both contain only placeholders.
4. Rehydrate. When the model returns, swap placeholders back to the real values inside your trust boundary, before showing it to the user.

# stateless mode: nothing persists on our side
from peyeeye import shield

red = shield.redact(note, session="stateless")
# red.text:           '[PATIENT_1] visited on [DATE_1]'
# red.rehydration_key: 'skey_...'  (AEAD-sealed, opaque)

answer = openai.chat(red.text)
final = shield.rehydrate(answer, key=red.rehydration_key)
# the LLM saw placeholders. peyeeye stored nothing.

The thing that matters in that snippet, from a compliance angle, is what doesn't appear: a database write, a log line with the raw note, a token map that lives anywhere longer than the request.

What auditors actually look at

We've been on the engineering side of a few of these reviews. The questions that get asked most often, in roughly the order they get asked:

• Prompt logs, both sides. Show me a sample of the prompts you logged in the last 24 hours. Show me what the LLM vendor has on their side for the same window. Identifiers in either log set is a finding.
• The DPIA. A Data Protection Impact Assessment for a feature that processes special-category data. It names the data flows, the lawful basis, the mitigations, and the residual risk. Redaction goes in the mitigations column.
• Retention timers. For every place that holds data, what's the timer, and is it actually enforced. “We delete on a cron” with no proof is not enforced.
• Encryption at rest of any token map. If you store mappings, the store is encrypted with a key you control, and key rotation is documented.
• Key management. Where the sealing key for any AEAD blobs lives, who can read it, how it's rotated, and what happens to old blobs after rotation.
• Sub-processor list. The named vendors in the data flow, and which agreements (DPA, BAA, SCCs) are in place with each.

A short version of that list, for engineering: log nothing you don't need, store nothing you don't have to, encrypt what you do store, and write down where every piece of the flow lives. The reviewer's job is to translate that into the regulator's vocabulary; your job is to make the translation easy.

Cross-border transfers and Schrems II

One more thing that comes up in any EU LLM review: the data leaving the region. After Schrems II, transfers of personal data out of the EU need a transfer mechanism (Standard Contractual Clauses are the common one), an adequacy decision, or a more exotic arrangement. Most major LLM vendors are US-based, which means the question is on the table by default.

Redaction doesn't make Schrems II go away, but it changes the math. If the prompt leaving the EU only contains placeholders and the mapping stays inside the EU, the payload that crosses the border isn't identifiable on its own. That's a weaker statement than “no personal data left the region”, and your DPO will want to phrase it carefully, but it's a real argument for reducing risk under the SCCs. Some teams pair this with an EU-region inference endpoint when the vendor offers one, so the redacted payload doesn't leave the region at all.

Where redaction tools differ on the compliance axis

One last note: not all redaction tools are equivalent for this conversation. We've written about how peyeeye compares to Microsoft Presidio from an engineering angle, and the same comparison shifts when compliance is the lens. A self-hosted detector you ran inside your VPC is a strong story (no third-party processor in the path), but you've taken on the operational and accuracy burden yourself. A managed redaction API with zero retention is a different story: you've added a processor, but you've also offloaded the “is this thing actually working” question to a team whose job is keeping it working.

Both are defensible. Pick the one your team can actually maintain in 18 months, then write the DPIA around the choice you made.