PII, privacy and the EU AI Act
The compliance layer. Increasingly asked, and as of August 2026 no longer hypothetical — the AI Act’s transparency obligations are in force now.
Verified 2026-08-09. Regulatory timelines shift; re-check before relying on specifics.
PII in an LLM pipeline
PII leaks in more places than people plan for:
| Where | Risk |
|---|---|
| Prompts sent to a provider | data leaves your boundary |
| Provider-side logs | retention outside your control |
| Your own logs and traces | prompts and responses often logged verbatim |
| Checkpoints | durable agent state persists full message history |
| Vector index | embedded chunks contain the source text’s content |
| Fine-tuning data | baked into weights, not deletable |
| Model output | the model repeats PII it was given |
Checkpoints and fine-tuning data are the two that get missed. A durable agent persists conversation history to a database — that’s a PII store, subject to the same retention and deletion rules as any other. And PII in fine-tuning data cannot be removed without retraining, which makes a deletion request effectively unsatisfiable. Don’t fine-tune on personal data.
Handling
# Redact before the model sees it, restore after
redacted, mapping = redact_pii(text) # "John Smith" -> "[PERSON_1]"
response = llm(redacted)
final = restore(response, mapping)Placeholder-and-restore preserves utility while keeping raw identifiers out of the provider’s context and logs. Detection uses Presidio, spaCy NER or a small classifier — all imperfect, so treat it as risk reduction rather than a guarantee.
Also: don’t log prompts and responses verbatim by default. Log token counts, latency, cost, model version and a trace ID. Log content only when sampled, redacted, and with a short retention.
Data residency
If personal data must stay in a region, use a deployment that guarantees it — Bedrock, Azure OpenAI or Vertex in the right region — rather than a public API endpoint. This is usually the actual reason enterprises pick a cloud-hosted model over a direct provider API. See ../../backend/20_cloud_azure/01_azure_overview.md.
Check the provider’s terms on training from your inputs. Business tiers generally exclude it; consumer tiers may not.
The EU AI Act
Risk-tiered regulation with extraterritorial reach — it applies if your system is used in the EU, wherever you are.
| Tier | Examples | Obligation |
|---|---|---|
| Prohibited | social scoring, manipulative techniques, most emotion recognition at work | banned |
| High-risk | employment, credit, education, essential services | conformity assessment, risk management, logging, human oversight |
| Limited risk | chatbots, generative content | transparency |
| Minimal | spam filters, most business tooling | none |
What actually applies right now
The timeline was amended on 16 June 2026, which is the detail that separates a current answer from a stale one:
- Most high-risk obligations were delayed. Standalone high-risk systems (Annex III) moved from August 2026 to December 2027; product-embedded high-risk systems (Annex I) moved to August 2028.
- Article 50 transparency duties were NOT delayed and took effect 2 August 2026.
- GPAI enforcement powers and the penalty regime also activated 2 August 2026. The first year was compliance on paper without penalty exposure; that changed.
So the practical position today: transparency obligations are live, penalties are live, and the heavy high-risk conformity work has more runway than originally scheduled.
Article 50 — what you must do
Applies to most people building chat and generative features:
- Disclose that users are interacting with an AI, unless it’s obvious.
- Mark AI-generated content in a machine-readable way.
- Label deepfakes and synthetic media.
Pre-existing systems have until 2 December 2026 for the machine-readable marking duty specifically.
Practically: a visible “AI assistant” label, provenance metadata (C2PA-style) on generated media, and disclosure in the UI rather than buried in terms.
If you’re high-risk
Risk management, data governance, technical documentation, automatic logging of operation, human oversight, accuracy and robustness measures, and conformity assessment before market.
The engineering implication is that audit logging and human oversight stop being nice-to-haves. If your system decides anything about employment, credit or access to essential services, you need to demonstrate a human could meaningfully intervene, and you need records proving what it did.
Other regimes
- GDPR still applies independently. Automated decisions with legal or similarly significant effects trigger Article 22 rights, including meaningful information about the logic involved — which is a real argument for interpretable models in those domains. Right to erasure collides with fine-tuning, as above.
- Sector rules — HIPAA, PCI-DSS, financial regulation — apply as they always did. An LLM in the flow doesn’t exempt anything.
- Copyright on training data and outputs remains contested. Note it as a risk, don’t claim certainty.
What to build
| Requirement | Implementation |
|---|---|
| Disclosure | visible AI labelling in the UI |
| Content marking | provenance metadata on generated media |
| Audit trail | immutable log of inputs, outputs, model version, decision |
| Human oversight | approval gates, override capability, escalation path |
| Data governance | retention policy, deletion path, residency controls |
| Explainability | log retrieved context and reasoning traces |
The audit trail is the one to mention. “Every decision is logged with its inputs, the model and prompt versions, and the retrieved context” is both a compliance answer and good engineering. See ../15_mlops_llmops/.
Interview angle
- “What does the EU AI Act require of a chatbot?” — Article 50 transparency: disclose it’s an AI, mark generated content machine-readably, label deepfakes. Those took effect 2 August 2026 and were explicitly not delayed, unlike most high-risk obligations.
- “Wasn’t the AI Act delayed?” — partly. June 2026 amendments pushed standalone high-risk obligations to December 2027 and product-embedded ones to August 2028. Transparency duties, GPAI enforcement powers and the penalty regime all still started on 2 August 2026.
- “Where does PII leak in an LLM system?” — prompts to the provider, provider and your own logs, agent checkpoints persisting message history, the vector index, fine-tuning data, and the output itself. Checkpoints and fine-tuning data are the ones usually missed.
- “A user requests erasure. Can you comply?” — for prompts, logs, checkpoints and the vector index, yes with a deletion path. For anything fine-tuned into weights, no — which is the argument for never fine-tuning on personal data.
- “How do you use an LLM with data that can’t leave the region?” — a region-pinned managed deployment (Bedrock, Azure OpenAI, Vertex) rather than a public endpoint, plus checking the provider’s terms on training from inputs. That requirement is usually why enterprises choose cloud-hosted models.
- “What changes if your system is classified high-risk?” — risk management, data governance, technical documentation, automatic operational logging, demonstrable human oversight and conformity assessment. Engineering-wise: audit logging and a meaningful human override become mandatory rather than optional.