Privacy in the application layer: an in-context encryption protocol for LLM APIs

1. Introduction

Modern AI features rarely run where the data lives. A typical production workflow serializes user records, support tickets, or internal documents into a prompt and sends it to a hosted model over a third-party API. Providers offer contractual controls for this — no-training clauses and zero data retention (ZDR) agreements — but while integrating LLM functionality into production-adjacent workflows, we observed that ZDR is not always implicitly active. It must be verified explicitly, per provider, per endpoint, and per contract.

Even where retention guarantees hold, a request traverses infrastructure that can log plaintext verbatim: API gateways, routing layers, observability tooling, and application middleware. “No training” is a statement about how weights are updated; “no retention” is a statement about storage. Neither is observable, let alone enforceable, from the application side.

This raises a concrete question: can we establish a protocol under which sensitive fields remain unreadable even if every message in the exchange is logged verbatim? The short answer is yes — with an important caveat about model cooperation, which turns out to be the binding constraint.

2. Threat model

We assume an honest-but-curious channel: the provider and all intermediate infrastructure execute the task faithfully, but any component may store the messages it sees. The protocol aims to guarantee confidentiality of designated sensitive fields against any party that reads those logs after the fact. It does not attempt to hide information from the model at inference time beyond the masked fields themselves, and it does not defend against an actively malicious provider that alters model behavior.

3. The protocol

We implemented the protocol as a Python library that wraps the chat-completion call, so that masking and unmasking are transparent to the calling application. An exchange proceeds in three stages:

1. Stage 1 — Key establishment. The client and the model agree in-context on an ephemeral, reversible masking scheme: a keyed substitution over the sensitive entities in the conversation. The key material is generated client-side and is never disclosed in a reusable plaintext form.
2. Stage 2 — Masked task execution. Before transmission, the library replaces sensitive fields with cipher tokens. The model is asked to perform the actual task — extraction, transformation, reasoning — directly over the masked prompt.
3. Stage 3 — Response masking. The model must keep cipher tokens masked in its output. The library then reverses the substitution locally before returning the result to the application.

4. Experimental setup

The resulting security property is simple to state: a verbatim log of the exchange, taken anywhere between the client and the model, contains only cipher tokens. The mapping required to reverse them exists only in client memory.

To establish whether the concept is technically feasible with current frontier models, we evaluated protocol compliance on three models accessed through their public APIs: openai/o3, anthropic/claude-opus-4-8, and google/gemini-3.1-pro-preview. Each stage was run in three independent trials. A trial passes if the model completes the stage without leaking plaintext, breaking the scheme, or refusing to proceed. Scores are therefore counts of passed trials out of three per stage.

5. Results

All three models completed key establishment in every trial: no model objected to agreeing on a masking scheme. Behavior diverged once models were asked to operate on and reproduce masked content.

openai/o3 completed all nine trials, maintaining the substitution scheme consistently through multi-step tasks. google/gemini-3.1-pro-preview passed eight of nine, with a single failure in masked task execution where it partially reverted cipher tokens to plaintext mid-response.

anthropic/claude-opus-4-8 presents the most interesting case. It established keys in all three trials but declined to produce masked output in every trial of stages two and three, stating that masking its responses conflicted with its guidelines. The refusals were consistent rather than stochastic — zero passes in six trials — which suggests an alignment-layer policy rather than a capability failure.

6. Discussion

Feasibility. The results indicate that application-layer confidentiality protocols are technically feasible today: two of three frontier models sustained the full scheme with high reliability, with no changes to provider infrastructure and no special API access.

Alignment as a protocol constraint. The failure mode we observed in claude-opus-4-8 is not a security failure but a policy collision. Safety training that treats deliberately obfuscated text as a risk will also reject a privacy measure that depends on obfuscation. In effect, a provider’s alignment layer decides which security protocols an application is permitted to implement — a dimension of model choice that, to our knowledge, receives little attention in procurement decisions.

The practical implication is the conclusion we started from: “no training” and “no retention” are not the same thing, and neither is verifiable by the client. Privacy-by-design has to live in the application layer, where it can be enforced and audited by the party that owns the risk.

7. Limitations

In-context masking is not semantically secure encryption in the cryptographic sense; a keyed substitution over entities leaks structure, and its strength depends on the entity granularity chosen by the client. Our sample is small — three trials per stage — and model compliance is probabilistic and version-dependent, so the scores should be read as a feasibility snapshot from June 2026 rather than a stable benchmark. Finally, masking can degrade task quality when the task genuinely requires the semantics of the hidden values; quantifying that cost is future work.

8. Conclusion

We asked whether sensitive data can be sent to hosted LLMs in a form that stays unreadable in any log along the way, and found that current frontier models can sustain such a protocol — when their alignment policies permit it. Confidentiality guarantees offered contractually should be treated as complements to, not substitutes for, controls the application enforces itself. We plan to extend the evaluation to a broader model set, formalize the security properties of the masking scheme, and measure the task-quality cost of operating over masked data. If you are working on similar problems, we would like to hear from you.