A context window is the maximum amount of text a language model can take into account in a single request — your instructions, the conversation so far, any attached files, and the answer it is writing, all counted together. It is measured in tokens rather than words: roughly three-quarters of a word each for ordinary English, so a 200,000-token window is on the order of 150,000 words.
The part that surprises people: models have no memory
A model does not remember your previous message. The computation that answers a request retains nothing once it is finished. Conversation only feels continuous because the earlier turns are supplied again — most often by the client re-sending the whole transcript, so the model reads it all again every time and writes the next reply. Some APIs offer the other arrangement, where the provider keeps the thread on its own servers and the client refers back to it by id. That is more convenient and less private: it means the conversation is being stored somewhere by design.
So the context window is not a memory budget that fills up once. It is a per-request ceiling, and a long conversation approaches it because the conversation itself is the input. When a chat outgrows the window, something has to give: older turns get dropped or summarised, which is why a very long thread can start "forgetting" its own beginning.
Why it matters for privacy
If every turn re-sends the transcript, then every turn re-exposes it. A sensitive detail mentioned once in message three is not sitting quietly in the past — it is part of the payload sent to the provider again with message four, and message five, for as long as it stays inside the window. This is the mechanical reason behind advice that otherwise sounds fussy: start a new chat when you change subject, rather than carrying an old thread forward.
It also means "the model does not remember me" and "my data was not sent" are different claims. The first is true of the model itself; the second depends on what goes into the request, whether the provider keeps the thread on its side, and what it does with either afterwards — which is the subject of zero-retention APIs and training opt-outs.
Cost, latency and the window
Input tokens are billed, so a long conversation costs more per turn than a short one even when your question is identical — you are paying to re-send the history. Several providers price a long request at a higher rate once it crosses a threshold, and cached input is usually cheaper than fresh input. Very long contexts also slow a response down and, in practice, can dilute quality: a model given 100,000 tokens of history does not attend to all of it equally well, and a focused prompt often beats an exhaustive one.
A worked example from this app: a follow-up question after an AI Council turn carries every panellist's full answer as conversation history, not just the referee's summary — otherwise the next model would see a verdict with none of the material behind it and could not be asked why one model disagreed. That is deliberate, and it makes that particular turn several times the size of an ordinary one.
Frequently Asked Questions
- What happens when a conversation exceeds the context window?
The request cannot be answered as-is, so something has to give: the client shortens it by dropping the oldest turns or replacing them with a summary, or the API truncates it server-side, or the request is simply rejected. Where the history is trimmed, the visible symptom is a model that no longer recalls something you established early in a long thread.
- Is a bigger context window always better?
No. It removes a hard limit, which is genuinely useful for long documents, but it costs more, takes longer, and does not guarantee the model uses the middle of a long input as well as it uses the ends. For most questions a short, specific prompt produces a better answer than a large one.
- Does the context window mean the AI remembers me?
Not by itself. Within one request the model sees whatever was sent; between requests it retains nothing. Anything that persists across separate conversations is a product feature built on top — a stored memory or profile — which is a separate thing to check and usually a separate setting to turn off.
- Do attachments count against the window?
Yes. A PDF or an image has to be represented as tokens for the model to read it, and those tokens come out of the same budget as your text. A large attachment can consume most of a window on its own.
Related terms: System Prompt · AI Hallucination · all glossary terms