Identity, Privacy, Retention, and Deletion

On 31 May 2023 the United States sued Amazon over Alexa in the Western District of Washington. The complaint describes the shared-household failure as a design rather than an accident. More than 800,000 children under 13 had their own Alexa profiles. Each was linked to a parent's profile on shared household devices. The identity boundary was drawn around the device and the account that owned it. Not around the person speaking to it.

The second half of the complaint is about deletion. Until mid-2019, the United States alleges, Amazon honoured a parent's deletion request against the audio and kept the writing: “Specifically, when users requested deletion of their voice recordings, or when parents used the Alexa Privacy Hub or the Alexa App to delete their children’s voice recordings, Alexa deleted the voice recordings but retained written transcripts of those recordings in residual data stores.” Those transcripts went on being used for product improvement.

The case was terminated on 20 July 2023 and settled for a $25,000,000 civil penalty. Both halves of the lesson sit in one filing. An identity boundary set by hardware, and a deletion that reached the record the user pointed at and stopped there.

Agent identity can refer to end users, accounts, devices, organizations, service principals, or delegated actors. Memory has to be partitioned by the identity and the purpose that justify collecting it and reading it back. Partitioned means the agent cannot read across the wall. What it learned for one account is unavailable while it works for another.

Deletion should cover primary records and derived indexes, summaries, caches, and embeddings according to documented timelines. That is not a design aspiration. It is the operative part of an order in force. The Federal Trade Commission's order against Everalbum, issued 6 May 2021, put three clocks on one request: photos and videos of deactivated users within 30 days, all non-consented Face Embeddings within 90 days, and any “Affected Work Product” within 90 days. The order defines that last term itself: ““Affected Work Product” means any models or algorithms developed in whole or in part using Biometric Information Respondent collected from Users of the “Ever” mobile application.” The Commission's public summary lined the three items up: “Part III of the proposed order requires Respondent to delete (A) photos and videos of Ever app Users who requested deactivation of their accounts, (B) face recognition data that it created without obtaining Users’ affirmative express consent, and (C) models and algorithms it developed in whole or in part using images from Users’ photos.”

Each deletion had to be confirmed to the Commission in a statement sworn under penalty of perjury. The order also carries an express carve-out. Retention is permitted where a law, regulation, court order or the safeguarding of evidence in pending litigation requires it. Embeddings named, models named, days counted, exception written down. That is what a retention policy looks like when someone else drafts it for you.

Identity boundary, Purpose boundary, Access policy, Retention policy, and Correction and deletion are decisions taken before personalization. Not after it. They are also not this lesson's invention. Each one has an identifier in the NIST Privacy Framework, Version 1.0, published on 16 January 2020.

The identity boundary is ID.IM-P3: the categories of individuals whose data are processed are inventoried. The purpose boundary is ID.IM-P5, which the Core states in a single line: “ID.IM-P5: The purposes for the data actions are inventoried.” Correction is CT.DM-P3, data elements can be accessed for alteration. Deletion is CT.DM-P4, data elements can be accessed for deletion. The retention schedule is CT.DM-P5, data are destroyed according to policy.

Inventoried is the load-bearing word in both identity subcategories. An inventory is a list you can be handed and audited against. It is what the first two boundaries produce, and it is why they cannot be reconstructed later. The first boundary says whose data this is. The second says what it was gathered for. Neither can be added after the agent has already learned from everything. And a reader who has to argue the point with a procurement team has the citations to argue it with: ID.IM-P3, ID.IM-P5, CT.DM-P3, CT.DM-P4 and CT.DM-P5.

Device-level memory, Account-scoped memory, and Task-scoped state answer the question of whose data this is. Each one answers it more precisely than the last.

Device-level memory follows a shared device or installation. It is convenient, it is ambiguous about identity, and it carries the highest disclosure risk. The Alexa complaint is what that risk looks like once it is written down as an allegation: the profiles of more than 800,000 children under 13, linked to a parent's profile on shared household devices. Account-scoped memory ties data to an authenticated account or tenant. That is a clearer partition. It is still exposed to shared accounts, and it depends on delegated roles being modelled properly. Task-scoped state exists only for one bounded workflow: strong minimization, limited continuity, and the safe default when you are not sure which of the other two you have earned.

A scope is only real if cross-user and cross-tenant retrieval is blocked by design rather than by convention. The boundary has to be enforced where the data is read, not left to the prompt. Deletion is the harder half of the same question, and the two Alexa filings show why. An identity boundary drawn at the device made the retention question unanswerable per person. The retention failure then survived into the transcripts and the models.

A governance matrix makes the identity-by-purpose grid visible, and it should be built against one real workflow. Map identities: users, delegates, organizations, devices, services, and shared contexts. Map purposes: tie every memory class to a permitted future use. Set access scopes across tenant, role, tool, and agent. Define retention, with expiry, correction, legal hold, and archival behavior. Then test deletion end to end. Remove a record and verify that indexes, summaries, caches, and outputs no longer use it. One row per kind of user, one column per purpose, and a rule in every cell that somebody has to sign.

The last step has a limit, and the limit belongs in the cell rather than in a footnote. The same model parameters can be reached from different datasets. So an entity can satisfy the standard approximate-unlearning definition without changing the model at all. Thudi and three co-authors showed that in 2022, and their abstract states the consequence flatly: “Our results show that even for a given training trajectory one cannot formally prove the absence of certain data points used during training.”

So a cell cannot honestly say the data is gone from the model. What it can say is which specific deletion algorithm was run, and where its audit trail lives. Those four authors argue that an algorithm designed for external scrutiny is the only auditable claim available. Filling the cells is what surfaces that awkwardness. Where the rows stay separate, and no cell lets one row read another's data, the matrix is doing its job.

Privacy controls belong in the memory architecture before personalization begins. Retrofitting identity and deletion after deployment is difficult and incomplete. The difficulty has been measured.

Sharding the training data is one way to pay for deletion in advance. SISA does exactly that: each point influences only one constituent model, so forgetting a point costs one shard rather than everything. Bourtoule and seven co-authors published it in 2021, and their abstract puts a number on it: “Under no distributional assumptions, for simple learning tasks, we observe that SISA training improves time to unlearn points from the Purchase dataset by 4.63×, and 2.45× for the SVHN dataset, over retraining from scratch.” For ImageNet classification the improvement was 1.36×.

That 4.63× is not a deletion feature anyone shipped after a complaint arrived. It is the return on a partitioning decision taken before a single point was trained on. It is what "privacy controls belong in the memory architecture before personalization begins" looks like in numbers. Whoever answers a deletion request is the person who has to chase the copies. Give that person a way to find everything the deleted fact was used to build. Governance holds when the system itself enforces the separation between users and tenants, and when deletion reaches the derived copies as well as the original.