Conversation State, Memory, and Privacy Boundaries

They persist for different durations and support different controls, and only two of them are records of a person at all.

Current context is the serialized turns and evidence included in the active request. It is visible to the model now, it consumes token budget, and it can be pruned or reordered. It does not persist unless the application stores it.

The KV cache is the one most often misdescribed. It is runtime attention state that avoids recomputing prior token projections during active generation. The systems literature treats it as a memory-management problem, not a semantic one. The 2023 paper that introduced vLLM opens its abstract with the difficulty: “However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically.” Kwon and eight colleagues wrote it. Their remedy is paging that memory, and it raises throughput by 2–4x at equal latency against FasterTransformer and Orca.

That is the register the KV cache belongs to. Per-request serving memory, bound to model execution, sized by the sequence, discarded with it, and measured as a throughput multiple. It is not a user profile. It is not durable application memory.

Session state is different in kind. It is application records for the current workflow or conversation, tracking entities, decisions and open tasks. It can be expressed in typed schemas, it needs expiration and correction, and it often survives several model calls.

Summary memory is a compressed representation of earlier interactions. It controls context growth. It can introduce omission or distortion, it needs source links and refresh, and it should not become unquestioned truth.

A durable profile is the only one that is a long-lived claim about a person: preferences or facts stored for future sessions, requiring purpose and user control, liable to become stale or sensitive, needing an access and deletion policy, and prone to unsupported inference. The others describe a conversation. This one describes a person.

A travel assistant remembers that a user once requested wheelchair access for a relative. Months later it treats the user as disabled in unrelated conversations. Five distinct faults are stacked in that one stored row.

Models can propose that a detail may be useful later. The application should decide whether it is eligible for storage. Eligibility can depend on user request, purpose, sensitivity, confidence, source, and retention period.

Store the smallest useful fact. Distinguish user-provided statements from model inferences. High-impact inferred attributes should usually require explicit confirmation or remain unstored. The next two sections give the two reasons this is not merely tidy engineering. What is stored can be pulled back out. What is compressed can quietly change.

Create controls before storing the first persistent user fact. Six steps. The last three are not advice — there is a record behind each of them.

1. Define eligible purposes. List the workflows that may read each memory class, so a fact collected for one task cannot silently serve another.

2. Classify sensitivity. Separate ordinary preferences from identity, health, finance, and protected data — including the data of children, which carries its own statute.

3. Require a source. Record whether each stored item is a user statement, verified account data, or a confirmed inference.

4. Set retention and expiry. Use time limits and refresh rules appropriate to the fact. Indefinite is a choice, and it is an actionable one. On 31 May 2023 the FTC and DOJ filed United States v. Amazon.com, Inc., alleging that Amazon kept children's Alexa voice recordings indefinitely and, even after parents asked for deletion, failed to delete the transcripts from all of its databases. Amazon agreed to a $25 million civil penalty. It agreed to delete inactive child accounts along with certain voice recordings and geolocation data, and it was barred from using that data to train its algorithms. The docket closed on 20 July 2023. Samuel Levine, Director of the FTC's Bureau of Consumer Protection, put the retention rule in one line: “COPPA does not allow companies to keep children's data forever for any reason, and certainly not to train their algorithms.”

5. Provide inspection and correction. Letting users see, edit, or delete durable memory is not a product preference. It is Regulation (EU) 2016/679, adopted 27 April 2016. Article 16: “The data subject shall have the right to obtain from the controller without undue delay the rectification of inaccurate personal data concerning him or her.” Article 17(1)(a) adds the right to erasure where the data are no longer necessary for the purposes for which they were collected. That is the stale travel preference from earlier in this lesson, described as an obligation.

6. Test isolation and deletion across tenants, logs, indexes, backups, and downstream copies. Both halves of that step have a case attached.

Isolation first. On 20 March 2023 a bug in the open-source redis-py client let ChatGPT users see titles from other active users' chat history. It also exposed payment-related data of 1.2% of ChatGPT Plus subscribers active during a nine-hour window, 1 a.m.–10 a.m. Pacific: first and last name, email address, payment address, credit card type, expiry date, and the last four digits only of the card number. Italy's Garante per la protezione dei dati personali recalled that breach in its press release of 31 March 2023 announcing the measure: “ChatGPT, il più noto tra i software di intelligenza artificiale relazionale in grado di simulare ed elaborare le conversazioni umane, lo scorso 20 marzo aveva subito una perdita di dati (data breach) riguardanti le conversazioni degli utenti e le informazioni relative al pagamento degli abbonati al servizio a pagamento.” Note the boundary carefully. The underlying provvedimento of 30 March 2023 ordered the immediate provisional limitation of OpenAI's processing of the data of subjects established in Italy. It does not itself mention the breach. It rests on four other things: the absence of any privacy notice, the absence of a legal basis for training data, the inaccuracy of ChatGPT's output, and the absence of age verification (Articles 5, 6, 8, 13 and 25 GDPR).

Then downstream copies. In the Everalbum matter, announced 11 January 2021, the FTC alleged that the company retained the photos and videos of users who had deactivated their accounts indefinitely. The analysis of the proposed consent order, published on 25 January 2021, set out how far deletion had to reach: “Part III of the proposed order requires Respondent to delete (A) photos and videos of Ever app Users who requested deactivation of their accounts, (B) face recognition data that it created without obtaining Users' affirmative express consent, and (C) models and algorithms it developed in whole or in part using images from Users' photos.” Deletion that stops at the primary table is not deletion. Clause (C) is the standard.

Conversations go better when the application preserves task state and user choices on purpose. The same persistence creates privacy, bias, and stale-assumption failures, and the price list for those failures is now public. $200 to extract 10,000 memorised examples from a deployed assistant. A $25 million civil penalty for keeping children's recordings and missing them at deletion time. A consent order that reached the models and algorithms trained on the data, not only the data itself.

Retrieval-augmented generation begins in the next lesson. It provides another form of external context. Its documents and indexes need their own provenance and lifecycle — the same six steps, applied to a corpus instead of a profile.