Is my AI conversation end-to-end encrypted?

No mainstream AI assistant is, in the sense a messaging app means it. Your prompt travels over TLS, which protects it from anyone between you and the provider, and is then decrypted at the provider's edge — because the model must read the plaintext to answer it. Encryption in transit says nothing about what happens after arrival.

Does the model remember my previous messages?

Not by itself. The classic chat endpoints are stateless — Anthropic describes its Messages API as handling "stateless multi-turn conversations", where prior turns are supplied in the request — so chat apps rebuild the transcript and resend it every turn (some truncate or summarise older turns to fit the context window). A sensitive sentence in your third message, in a chat that reaches forty of your messages, therefore crossed the network 38 times. Newer server-side options such as OpenAI's Responses API keep the conversation on the provider instead, which stops the resending by holding the transcript for you. Either way the model's weights do not change when you talk to it.

Is my prompt processed on its own, or alongside other people's?

On a busy shared endpoint, alongside. Mainstream inference servers use continuous batching (NVIDIA calls it in-flight batching) to step many requests forward together on the same hardware, and vLLM's PagedAttention allows "flexible sharing of KV cache within and across requests" — deliberate reuse of identical prefixes, not one user's context becoming visible to another. Separation between requests is enforced by the serving software; it is not a dedicated machine per user.

Do providers keep my prompt in a cache after the request finishes?

They keep something computed from it, above a minimum prompt length — not your text. OpenAI describes cached prefixes as "encrypted key/value tensors in GPU-local storage", and Anthropic matches cache entries by a cumulative hash of the prefix. The timings are documented: for GPT-5.6 and later a prefix "remains eligible for reuse for 30 minutes after its most recent write or reuse" (OpenAI notes it may retain the entry longer), earlier models run 5 to 10 minutes of inactivity up to an hour, and on models supporting both policies organizations without Zero Data Retention default to 24-hour retention. Anthropic's cache has a 5-minute default lifetime with an optional one-hour duration. Both state that caches are never shared between organizations — a property that matters, because a February 2025 Stanford audit found seven providers sharing caches globally across users and demonstrated a response-timing side channel against them.

If I delete a chat, is my prompt gone from the provider?

Not necessarily, and what else it reaches depends on the provider and the product. Deletion addresses the stored transcript the product wrote. Records kept on separate schedules generally are not covered — abuse-monitoring logs (OpenAI: up to 30 days by default, longer where required by law or reasonably necessary to prevent harm), trust-and-safety classification scores (Anthropic: up to 7 years, with flagged content up to 2 years), billing and usage aggregates, analytics held by a vendor, cache entries expiring on their own timetable, and anything frozen under a legal hold.

Can zero data retention stop all of this?

It closes the largest fork, not all of them. Endpoints that exist to hold state — conversations, assistants, threads, vector stores — are ineligible for it. Safety carve-outs survive it: OpenAI documents retention and human review of classifier-flagged content under its Safety Retention arrangement, and Anthropic still retains user-safety classifier results to enforce its Usage Policy. And Anthropic's "covered models" carry 30-day retention on every platform, so a ZDR customer must enable retention in that workspace to use them at all.

What can Secret Chat AI actually change about this pipeline?

One stage of it: the identity attached at the front door. Your prompt still travels, is still decrypted, tokenized, batched, cached and logged — but the record on the provider's side carries our gateway's credentials and server address, not your name, account or IP. It does not remove the data from your messages; whatever you type arrives verbatim, so redacting identifying details before sending is still up to you.

Inside an LLM Provider: What Happens to Your Prompt

Published August 18, 2026 · Facts last verified August 18, 2026

This article is general information, not legal advice. Vendor terms and retention policies change quickly. For your own situation, consult a qualified lawyer or your compliance officer. Your use of Secret Chat is governed by our Terms of Service and Disclaimers.

You press Enter. About a second later, words start appearing.

In that second your sentence crossed an ocean, was decrypted by a machine that is not the model, attached to your account, counted, judged by a classifier, chopped into numbers, queued, computed on a chip it was sharing with strangers' sentences, written into a cache that will outlive the request, and — depending on which of half a dozen forks it took on the way out — copied into a place that may keep it for thirty days, two years, or seven.

Almost every article about AI privacy argues about policy: will they train on it, can a human read it, does delete really delete. Those are the right questions, but they are the downstream questions. Each of them is an answer a company gives about a machine that already exists, and the machine has a shape. Knowing the shape tells you which promises are cheap for a provider to make, which are expensive, and which are physically impossible no matter who is asking.

So this article does something different. It follows one prompt through the pipeline, stage by stage, and stops at each point where a copy can be made. Nothing here is a leak or a scandal. It is the ordinary architecture of a service answering billions of requests a day — which is exactly why it is worth understanding.

Stage 1: It Leaves Your Machine, and Stops Being Encrypted

Your prompt travels to the provider over TLS, the same encryption your bank uses. That protects it from everyone on the road: your café's Wi-Fi, your ISP, whoever is sitting between you and the data centre.

It protects it from nobody at the destination. TLS terminates at the provider's edge — a load balancer or CDN node that decrypts the request, reads the routing information, and forwards it inward. Internal hops may well be re-encrypted between services, but that changes the plumbing rather than the fact: from that point your sentence is readable wherever it is processed, and it has to be, because the model cannot answer a question it cannot read.

This is the single most misunderstood point in consumer AI privacy, so it is worth stating without hedging. No mainstream AI assistant is end-to-end encrypted in the sense that a messaging app is. There is no arrangement in which the provider holds only ciphertext and the answer appears anyway. Encryption in transit is real and necessary, and it says nothing whatsoever about what happens after arrival. Everything else in this article happens on the far side of that decryption.

The exception worth knowing about — and its limits. A different architecture does exist, and it is shipping, just not in the products this article is about. Confidential computing runs inference inside a hardware-attested enclave that the operator has deliberately built itself out of. Apple's Private Cloud Compute states that "user data is never available to Apple — even to staff with administrative access to the production service or hardware", that data "stays on the PCC nodes that are processing the request only until the response is returned", and that every production build is published to a cryptographically tamper-proof transparency log so researchers can verify the code that is actually running. Meta shipped Private Processing for WhatsApp on the same principle in April 2025, promising that "no one except you and the people you're talking to can access or share your personal messages, not even Meta or WhatsApp."

Two things about that. It is genuinely a different guarantee, and worth watching. And it is not what happens when you open a general-purpose AI chat product and pick a frontier model — those run the pipeline described below, which is why the pipeline is worth knowing in detail rather than waiting out.

Stage 2: The Front Door, Where Identity Attaches

Before the request goes anywhere near a GPU, it hits an ordinary web application: authentication, authorisation, rate limiting, quota accounting, routing.

Your session cookie or API key resolves to an account. That resolution is the most consequential event in the whole pipeline, and it happens before any of the interesting machinery starts. From this point on, the request is not a request — it is your request. The log lines written downstream, the classifier scores, the usage counters, the stored transcript: each is written against the identifier attached here, unless a provider has gone to deliberate trouble to keep them apart.

Nine more stages follow. This is the one that decides how much they matter. A timestamp joined to an account is a line in a file about a person; the same timestamp with nothing to join to is a load statistic. We took that idea apart in detail in what your prompts reveal about you — the short version is that metadata is inert until it has an anchor, and Stage 2 is where the anchor is welded on.

Stage 3: You Are Not Sending a Message. You Are Re-Sending the Conversation.

Here is the fact that surprises even technical users, because the chat interface is designed to hide it.

The model has no memory of your last message. The classic chat endpoints are stateless: Anthropic's documentation describes the Messages API as being used for "single queries or stateless multi-turn conversations", and explains that you specify "the prior conversational turns with the messages parameter". OpenAI's chat completions endpoint works the same way. There is no conversation living inside the model between your turns — there is a transcript, and it is submitted again in full every time.

Consumer chat apps rebuild that transcript for you on the server, which is why it feels like a conversation. It isn't. It is forty separate submissions of an ever-growing document — and what is submitted is more than you typed, since the product's own system prompt and tool definitions are prepended to it invisibly.

Two honest qualifications. Long conversations are not always resent whole: clients truncate, summarise or compact older turns to stay inside a context window. And the stateless design is no longer the only one on offer — OpenAI's Responses API can keep conversation state on the server, referenced by an identifier instead of re-uploaded each time. Note what that swaps rather than solves: the transcript stops crossing the network repeatedly because the provider is now the one holding it.

The privacy consequence is concrete and almost never spelled out. Suppose the third message you sent contained something you regret — a client name, a diagnosis, a real address. If the conversation runs to forty messages of your own, that sentence did not cross the network once. It crossed it thirty-eight times, was decrypted thirty-eight times, was screened thirty-eight times, and arrived as part of thirty-eight separate requests. Editing or deleting that one message in the interface does not reach backwards through any of it.

The practical rule that falls out of this is unglamorous and genuinely useful: long conversations keep re-transmitting their worst moment. When a chat turns sensitive, starting a fresh one costs you context and saves you thirty-seven redundant copies.

Stage 4: Your Words Stop Being Words

Next, the text is tokenized — broken by a subword scheme (byte-pair encoding, or a close relative) into fragments, each mapped to an integer. "Understandable" might become three tokens; a rare surname might become five; a common word is one. The model never sees a letter. It sees a list of numbers, and its entire understanding of your question is the statistical relationship between those numbers.

Two things follow, and they point in opposite directions.

The first is that tokenization is not a privacy measure, though it is regularly mistaken for one. For a given tokenizer it is deterministic, and the numbers decode straight back into your sentence — that is the entire job of the decoder sitting at the other end of the model. Not every provider publishes its production vocabulary, but a vocabulary nobody outside the company holds is not a protection either, since the company is the party holding the tokens. Tokens are a data format, not a disguise.

The second is that this is where your prompt acquires the number that follows it everywhere: its token count. Quotas are counted in tokens, invoices are counted in tokens, and rate limits are enforced in tokens, so the size of what you wrote is recorded in systems that have nothing to do with the model — the billing pipeline included. The words may be transient. The measurement of them is accounting data, and accounting data is kept.

Stage 5: The Safety Pass, Before the Model Ever Sees It

The major providers run automated classifiers around inference — ahead of it, alongside it, or over the output, and the arrangement varies by model, endpoint and customer configuration. Wherever it sits, it produces a structured artefact: a set of scores describing what a machine thought of your prompt across a list of risk categories.

That artefact is a separate record from the prompt, on a separate clock. Anthropic is unusually explicit about the numbers: it retains "inputs and outputs for up to 2 years and trust and safety classification scores for up to 7 years if your chat or session is flagged by our automated trust and safety systems as violating our Usage Policy". OpenAI documents an analogous "Safety Retention" arrangement — offered to customers otherwise approved for zero data retention or modified abuse monitoring — under which it may retain and human-review content that its classifiers detect as potentially violating its Usage Policies.

Read the ordering once more, because it is the clearest structural point in this article: on Anthropic's own published schedule, the ordinary conversation is deleted long before the classifier's opinion about it. The derivative record outlives the original — and the derivative record is the one already in machine-readable form.

What happens when a score crosses a threshold is not hypothetical. We wrote up the documented end of that path — classifiers running constantly, human reviewers one score away, a referral threshold the provider sets alone — in the Palm Beach case.

Stage 6: The Queue, and the Batch You Share With Strangers

Now the tokens reach the inference server, and something happens that almost no user pictures correctly.

Your prompt is not handed to a model that was waiting for it. GPUs are throughput machines, and serving one conversation at a time leaves most of that throughput unused. So the mainstream serving stacks — vLLM, TensorRT-LLM, SGLang and the rest — support continuous batching (NVIDIA calls it in-flight batching), and busy public endpoints run it. The scheduler assembles a batch of requests, steps the whole batch forward by one token, drops the sequences that finished, pulls waiting requests in, and repeats. NVIDIA's documentation describes the result plainly: "sequences in context phase can be processed together with sequences in generation phase."

In other words, at the moment your question is being answered on a busy shared endpoint, it is one entry in a batch whose other entries are other people's questions, from other accounts, arriving from other countries, moving through the same weights on the same hardware at the same time. (The real picture is messier than a tidy grid — production systems use ragged batches, chunked prefill, and often split the reading and writing phases across different machines — but the sharing is the part that matters here.) The vLLM paper that introduced PagedAttention, now widely adopted, goes further and describes "flexible sharing of KV cache within and across requests to further reduce memory usage" — which means deliberate reuse of identical prefixes and parallel samples, not one user's context becoming readable to another.

This is not a vulnerability, and it is not a reason to panic — it is how the economics of inference work, and providers treat the separation between neighbours as a boundary they are responsible for. But it is worth being accurate about what that separation is: your prompt's isolation inside the model server is a software property, not a physical one. There is no private instance of the model reserved for you; there is a scheduler, and the guarantee is that it was written correctly. Shared hardware is also exactly the setting in which side channels get found — as the next stage demonstrates.

Stage 7: The Cache — The Part That Outlives Your Request

As the model reads your prompt it builds a KV cache: the intermediate attention state for every token, held in GPU memory. Reading the prompt (prefill) is done in parallel and is compute-heavy, which is why a long prompt makes you wait for the first word; producing the answer is serial and mostly limited by memory bandwidth, which is why the words then arrive at a steady drip. Neither half is free, and on a long answer the drip is where most of the time goes.

Then comes the stage that matters most for this article, because it is the one place where something computed from your prompt is deliberately kept after your request has finished, and nobody calls it storage.

Recomputing an identical prefix for every request is pure waste, so providers cache it. The published parameters, as of this writing:

OpenAI. Caching applies to prompts above a minimum length — "1,024 tokens is a strict minimum" for GPT-5.6 and later, and between 1,024 and 2,048 tokens for earlier models. For GPT-5.6 and later, "the 30-minute lifetime begins when the prefix is written and refreshes whenever the prefix is reused." Earlier models keep cached prefixes "active for 5 to 10 minutes of inactivity, up to a maximum of one hour" — or, under an extended-retention configuration, "up to a maximum of 24 hours."
Anthropic. "By default, the cache has a 5-minute lifetime," refreshed each time the cached content is used, with an optional one-hour duration available. The minimum cacheable prompt varies by model and by platform — 512 tokens for the newest Claude models on Anthropic's own API (1,024 for the same models on Bedrock), and 1,024, 2,048 or 4,096 for others — and shorter prompts are simply processed uncached, with no error returned.

Two distinctions matter here, and skipping either produces a scarier or a cosier story than the truth.

Eligibility is not retention. The clock above is how long a prefix stays eligible to be reused — OpenAI's own framing for its newer models is that "a cached prefix remains eligible for reuse for 30 minutes after its most recent write or reuse", while noting that it may retain the entry longer than that. On the earlier models that support both policies, the physical default is the longer one: "organizations without Zero Data Retention enabled default to 24h." So the number you read in the performance documentation is the reuse window, not the erasure guarantee.

What persists is derived state, not a transcript. This is where a lot of writing on the subject — an earlier draft of this one included — quietly overstates the case. The cache does not hold your sentences. OpenAI describes cached prefixes as "encrypted key/value tensors in GPU-local storage" held as application state; Anthropic's cache is addressed by a cumulative cryptographic hash of the prefix, so a hit requires "100% identical prompt segments" and a single changed character produces a different hash and a miss. Nobody is browsing a folder of your prompts.

But derived state is still state. It has a location, a lifetime, and — this is the part that turned out to matter — an existence that can be detected from outside.

Both companies treat cache isolation as a security boundary and document it as such. OpenAI: "Prompt caches are not shared between organizations. Only members of the same organization can access caches of identical prompts." Anthropic: "Caches are isolated between organizations. Different organizations never share caches, even if they use identical prompts" — with the finer, per-workspace isolation applying on the Claude API, Claude Platform on AWS and Microsoft Foundry, while Bedrock and Google Cloud use organization-level isolation.

They document it because it was measured. In February 2025, Stanford researchers published Auditing Prompt Caching in Language Model APIs, which used response-timing differences — cached prefixes are answered faster — as a side channel. Auditing real APIs, they found seven providers sharing caches globally across users, meaning an attacker could learn that somebody else had recently sent a given prefix. The same timing analysis leaked an architectural detail that had not been public: that OpenAI's embedding model is a decoder-only Transformer. The per-organization isolation now written into both companies' documentation is the answer to exactly this class of problem.

The general lesson is bigger than caching. An optimisation is not neutral just because it was built for speed. Every layer that remembers something in order to go faster is a layer that remembers something.

Stage 8: Generation, One Token at a Time

The output is produced serially. The model computes a probability distribution over its whole vocabulary, the next token is chosen from it — sampled, or taken greedily, or proposed by a smaller model and checked, depending on the configuration — that token is appended to the context, and the process repeats, each new token pushed into the KV cache and streamed back as it appears. The illusion of typing is not a UI flourish; it is close to the actual rate of production.

This stage creates plenty of transient copies — output tokens, cache entries, buffers — but it is not where durable records are made, and it is the shortest-lived stage in the pipeline. The model finishes, the batch moves on, and the working memory is handed back for reuse (which is a matter of allocator behaviour rather than a scrubbing guarantee). The one thing that certainly does not happen is the part people most often fear: your prompt does not enter the weights. Inference does not update a model's parameters. Training on retained conversations is a separate pipeline with its own policy — the fork in Stage 9, not a side effect of asking a question.

Stage 9: The Forks — Where Copies Are Actually Made

The answer has been sent. Now the pipeline splits, and "how long do they keep it?" turns out to be the wrong question, because there is no single it and no single clock. There are several records on several schedules.

Abuse-monitoring logs. OpenAI's developer documentation, quoted in full because the exceptions are the interesting half: "By default, abuse monitoring logs are generated for all API feature usage and retained for up to 30 days, unless longer retention is required by law, or is reasonably necessary to protect our services or any third party from harm."
Application state, which is a different thing entirely. On OpenAI's API, zero-data-retention-eligible endpoints include chat completions, responses, images, embeddings, audio and moderations. Others are ineligible because holding state is the point of them: conversations, assistants, threads and vector stores keep what you put there until you delete it. (The video endpoint is ineligible for a different reason — it writes to disk while working, keeps the result 48 hours so you can download it, then 30 days for abuse monitoring.) The underlying rule is worth stating plainly: you cannot have zero retention and a memory feature. They are the same feature with opposite signs.
Safety carve-outs inside the exception. Even under zero data retention, OpenAI documents a "Safety Retention" arrangement covering content its classifiers flag, and an "Eyes Off" option under which retained content is excluded from human review "unless required by applicable law". Anthropic's zero-data-retention arrangements still retain user-safety classifier results in order to enforce its Usage Policy.
The retention you cannot configure away — you can only decline the model. Anthropic designates certain frontier models as "covered models" — "Mythos-class models and future models with similar capabilities" — and states that prompts submitted to, and outputs generated by, covered models "are retained for 30 days", on every platform where those models are offered. It is not that zero data retention is silently overridden: a customer holding a ZDR agreement has to enable retention in the relevant workspace in order to use these models at all, and can keep other workspaces as they were. The stated reason is safety work that only functions across requests — detecting jailbreak patterns and state-sponsored misuse that are invisible one prompt at a time. Whatever you make of the trade, note its shape: the strongest privacy configuration on the market and the most capable model are now, for that customer, a choice between two things rather than a stack of both.
Training, or not. On the paid API tiers this fork is generally closed by contract rather than by a toggle. Google's Gemini API terms for paid services: "Google doesn't use your prompts (including associated system instructions, cached content, and files such as images, videos, or documents) or responses to improve our products" — while still logging prompts and responses "for a limited period of time, solely for detecting and preventing violations of the Prohibited Use Policy". That logging also serves service safety and security and any legally required disclosure, and grounding with Google Search stores prompts, context and output for 30 days on its own account. On consumer tiers the polarity is usually reversed, and we mapped that split in consumer ChatGPT versus the API.
Analytics, which is a third party. Product telemetry is typically bought rather than built, which puts a copy of the envelope in a vendor's systems. The November 2025 Mixpanel incident at OpenAI exposed exactly that class of data — names, email addresses, approximate locations, device details — with no chat content involved at all. We catalogued it with the other incidents in what happens in a provider data breach.
Legal hold, which overrides all of the above. When a preservation duty attaches, ordinary deletion stops. OpenAI operated under exactly such an order from May 2025 until it ceased to apply to data generated after 26 September 2025 — and note that ending a hold is not the same as undoing it: what was captured while it ran stays preserved. The detail, and what was and was not swept into it, is in can your AI chats be subpoenaed.

And under all of it sits geography: which continent the request was stored in, and which continent it was computed in, are separate promises sold separately — see where your prompts physically live.

Stage 10: What "Delete" Reaches

Now put the delete button against that list.

Deleting a conversation is a real operation with a real effect on the record it was built for: the stored transcript in the product. What it reaches beyond that varies by provider, product and contract, and the honest general statement is narrower than the confident one — deletion addresses the record the delete button was built for, and the other records in Stage 9 run on their own clocks unless a provider says otherwise. Anthropic's published schedule is the clearest worked example: back-end deletion of ordinary conversations within 30 days, alongside multi-year retention of flagged content and its classification scores. Nor does deleting reach a cache entry, which expires on its own timetable regardless.

This is not a trick. It is the predictable result of a pipeline having ten stages and the delete button living at one of them. The mistake is in the mental model, not in the button: people picture deletion as reaching back along the path the prompt travelled, when it only ever addressed the destination the product wrote to.

The Pattern Underneath All Ten Stages

Read the list again and notice how few of the stages you can influence.

You cannot decline decryption at the point of processing — the model has to read the plaintext. You cannot skip tokenization. You cannot ask for a GPU of your own on a consumer plan, and you cannot switch off the cache. Those are properties of the machine rather than of your account, and a service claiming to have removed them is describing marketing rather than architecture. What a bigger contract buys is not a different machine: it is different settings on the same one — retention windows, isolation scope, whether a human may look, whether state is stored at all.

What actually varies between users is two things. The first is the wrapper: which tier you are on, what its contract says, which forks in Stage 9 are open. That is a real lever, it is worth pulling, and its ceiling is a better contract rather than a different pipeline.

The second is what the whole thing hangs from — the identity attached back at Stage 2. Every downstream record inherits it. Change it and you have not made any stage disappear; you have changed what all of them add up to. Ten stages of records about "the request that arrived at 14:32" are the exhaust of a machine. Ten stages of records about "the request you sent at 14:32" are a file.

How Secret Chat AI Fits — Including What Our Own Pipeline Does

Secret Chat AI is built around Stage 2 and nothing else, and an article about what really happens to a prompt would be worthless if it described only the flattering half of our own.

What changes. Your prompt reaches the provider through our gateway, under our credentials, from our servers. Stages 1 and 3 through 9 happen to it exactly as described above — we cannot and do not claim otherwise. What the provider's Stage 2 resolves is us, not you: no IP address of yours, no device, no browser, no account of yours, no payment identity. Your request joins the aggregate stream of everyone using the gateway. We build no profile of you, no conversation is ever associated with you, and queries are never used for training. Registration takes an email, but it is used only for account access and payment — never stored against your prompts. Retention may still apply at the provider; your query simply arrives anonymized. You use the model as a stranger.

What does not change — the sentence we will not soften. Secret Chat AI removes you from your queries; it does not remove the data from your messages. Your text reaches the provider's API verbatim. Write your own name, your client's name or your case number into a message and every word of it is still sitting there in the message. Redacting identifying details before sending remains your responsibility.

What our own gateway does with your prompt, plainly:

Your chats live in your browser, in IndexedDB and OPFS — there is no server-side chat archive. A prompt exists on our side only as a transient row, for as long as it takes to fetch your answer: it is deleted the moment your browser collects the reply, and an hourly sweep clears anything orphaned by a closed tab.
We run a moderation pass of our own. Before a prompt goes to the model it is screened by OpenAI's moderation classifier — so for a chat with a non-OpenAI model, that is a second provider seeing the text. We think a service with no screening at all is the worse answer, but it is a real trade and you should hear it from us rather than infer it.
Small helper models see some of your text. A chat's title is written by a cheap model — deliberately one from the same family you are already talking to, so a conversation you chose to keep with one provider is not sent to another just to name it. For image chats on providers whose API accepts only a single prompt string, a helper model compiles your conversation into one self-contained instruction; that step runs at OpenAI, and it is itemised in your Session Privacy Report rather than hidden.
We ask providers to delete, where a provider offers deletion. After an answer we call the provider's per-response deletion endpoint where one exists, and hand you a Session Privacy Report saying what was deleted, what was not, and why. Where a provider exposes no deletion API at all, the report says that too. Those report PDFs are archived durably on our side — they contain the model, the provider, the deletion status and response identifiers, and no prompt text.
Ordinary infrastructure telemetry — IP addresses, request timestamps, error codes — is kept for at most 30 days. That is not a record of what you asked.
We cannot switch off a provider's prompt cache, and Stage 7 applies to us as it does to everyone. Caches are isolated per organization, and to a provider our gateway is the organization — so cache reuse across our own users' identical prefixes is possible by design, and it carries no identity in either direction.

What we do not offer. No region pinning — where inference runs is the provider's default, not something you can select through us. We do not sign Business Associate Agreements and we are not a HIPAA or enterprise compliance solution. And anonymity is not privilege, not a legal exemption, and not a way to place anything beyond what a court is entitled to.

Frequently Asked Questions

Is my AI conversation end-to-end encrypted?
No mainstream AI assistant is, in the sense a messaging app means it. Your prompt travels over TLS, which protects it from anyone between you and the provider, and is then decrypted at the provider's edge — because the model must read the plaintext to answer it. Encryption in transit says nothing about what happens after arrival.
Does the model remember my previous messages?
Not by itself. The classic chat endpoints are stateless — Anthropic describes its Messages API as handling "stateless multi-turn conversations", where prior turns are supplied in the request — so chat apps rebuild the transcript and resend it every turn (some truncate or summarise older turns to fit the context window). A sensitive sentence in your third message, in a chat that reaches forty of your messages, therefore crossed the network 38 times. Newer server-side options such as OpenAI's Responses API keep the conversation on the provider instead, which stops the resending by holding the transcript for you. Either way the model's weights do not change when you talk to it.
Is my prompt processed on its own, or alongside other people's?
On a busy shared endpoint, alongside. Mainstream inference servers use continuous batching (NVIDIA calls it in-flight batching) to step many requests forward together on the same hardware, and vLLM's PagedAttention allows "flexible sharing of KV cache within and across requests" — deliberate reuse of identical prefixes, not one user's context becoming visible to another. Separation between requests is enforced by the serving software; it is not a dedicated machine per user.
Do providers keep my prompt in a cache after the request finishes?
They keep something computed from it, above a minimum prompt length — not your text. OpenAI describes cached prefixes as "encrypted key/value tensors in GPU-local storage", and Anthropic matches cache entries by a cumulative hash of the prefix. The timings are documented: for GPT-5.6 and later a prefix "remains eligible for reuse for 30 minutes after its most recent write or reuse" (OpenAI notes it may retain the entry longer), earlier models run 5 to 10 minutes of inactivity up to an hour, and on models supporting both policies organizations without Zero Data Retention default to 24-hour retention. Anthropic's cache has a 5-minute default lifetime with an optional one-hour duration. Both state that caches are never shared between organizations — a property that matters, because a February 2025 Stanford audit found seven providers sharing caches globally across users and demonstrated a response-timing side channel against them.
If I delete a chat, is my prompt gone from the provider?
Not necessarily, and what else it reaches depends on the provider and the product. Deletion addresses the stored transcript the product wrote. Records kept on separate schedules generally are not covered — abuse-monitoring logs (OpenAI: up to 30 days by default, longer where required by law or reasonably necessary to prevent harm), trust-and-safety classification scores (Anthropic: up to 7 years, with flagged content up to 2 years), billing and usage aggregates, analytics held by a vendor, cache entries expiring on their own timetable, and anything frozen under a legal hold.
Can zero data retention stop all of this?
It closes the largest fork, not all of them. Endpoints that exist to hold state — conversations, assistants, threads, vector stores — are ineligible for it. Safety carve-outs survive it: OpenAI documents retention and human review of classifier-flagged content under its Safety Retention arrangement, and Anthropic still retains user-safety classifier results to enforce its Usage Policy. And Anthropic's "covered models" carry 30-day retention on every platform, so a ZDR customer must enable retention in that workspace to use them at all.
What can Secret Chat AI actually change about this pipeline?
One stage of it: the identity attached at the front door. Your prompt still travels, is still decrypted, tokenized, batched, cached and logged — but the record on the provider's side carries our gateway's credentials and server address, not your name, account or IP. It does not remove the data from your messages; whatever you type arrives verbatim, so redacting identifying details before sending is still up to you.

Conclusion

The honest summary of an LLM provider's infrastructure is that it is neither sinister nor private. It is a high-throughput pipeline that decrypts what you send because it must, measures it because it bills for it, screens it because its own policies — and, in some places, its regulators — require it to, batches it with strangers because an idle GPU is a wasted GPU, caches it because recomputation is waste, and logs it because that is how anyone operates anything at scale. Every stage has a mundane engineering reason, and every stage is a place where a record can form.

Which is why the useful defensive question is not "do they keep it?" — they keep several different things on several different clocks, and the answer changes with every terms update. It is: what is all of it attached to? Ten stages of records about an anonymous request are the exhaust of a machine. The same ten stages, joined to an email, a phone number and a card, are a biography being written by a system that never set out to write one.

You can opt out of this pipeline only by leaving it — a model running on your own hardware, or one of the confidential-computing designs above, with the capability and convenience trade-offs each of those carries. Inside a mainstream AI product there is no setting that removes a stage. What there is, always, is the question of what the whole thing is allowed to know your name.

What Actually Happens to Your Prompt
Inside an LLM Provider's Infrastructure