Re-reading, every turn

A chatbot seems to remember what you said a few messages ago. The way it does this is simpler than it looks.

Every time you send a message, the whole conversation so far goes through the model again. Your messages, its replies and any instructions the app added all go in together.

The model has no other memory of the chat. What looks like remembering is re-reading.

The amount of text a model can take in at once is called its context window. It is measured in tokens, the pieces from Part 1.

A whiteboard of fixed size

Hello, AI pictures the window as a whiteboard. Everything the model can consider is what is written on the board right now. When the board fills up, old notes must be wiped to make room for new ones.

The model never feels the eraser. A person would remember that something had been there. For the model, the wiped notes never existed.

Imagine a tiny model that could see only the last 10 tokens. Tell it your mother loves Thai food, chat for three more sentences, and that fact has scrolled out of view.

Real windows are far larger. Hello, AI puts 2026 windows at about 200,000 tokens up to a million or more, enough for a long novel. At about three quarters of a word per token, 200,000 tokens is roughly 150,000 words. Scrolling off the edge is much rarer than it used to be.

The rule stays the same. If it is not in the window, it does not exist for the model.

Why windows have limits

Big windows are hard work. To relate words to each other, a model compares every token with every other token. Four times as many tokens means sixteen times the comparisons.

They also cost money. Services charge per token on every request, so a long document re-sent each turn is paid for each turn. On Anthropic’s October 2026 prices, filling a million-token window costs about $2 to $10, depending on the model, on every call. Caching text that repeats makes this much cheaper.

Hello, AI draws three habits from all this:

• A new chat starts from a nearly empty window, unless the app adds saved notes (more on that in Part 3).
• When a chat goes off the rails, start a fresh one and restate the task cleanly.
• If something matters, put it in the window yourself, in plain words.

In one sentence: the context window is everything the model can see right now, and whatever falls outside it is gone.