Privacy, Governance, and Audit Operations

A customer record is removed from the application database. The row is gone. Derived features remain in snapshots, training datasets, cached embeddings, model checkpoints, logs, and third-party evaluation exports — and the checkpoint is the one that gets argued about, because a set of weights does not look like a place where a person can be stored.

It has been shown to be one. In 2021 Carlini and colleagues attacked GPT-2 and pulled its training data back out. Their abstract: “We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data. These extracted examples include (public) personally identifiable information (names, phone numbers, and email addresses), IRC conversations, code, and 128-bit UUIDs.” The recovered sequences included ones that had appeared in just one training document. The larger models proved more vulnerable than the smaller ones.

A standards body has since classified the failure. NIST's January 2024 taxonomy of adversarial machine learning treats data reconstruction, memorization and membership inference as privacy attacks on deployed models, and ranks them: “Data reconstruction attacks are the most concerning privacy attacks as they have the ability to recover an individual's data from released aggregate information”.

So the checkpoint is an asset that holds personal data, not a summary that has forgotten it. Deleting a record for real takes an asset map, retention policy, propagation rules, and a decision about what can be removed, rebuilt, isolated, or lawfully retained.

A user invokes a data-rights workflow. Each stage below is a place where the request either propagates or quietly stops.

A complete document can exist inside a weak control system, and a mandated audit can exist while its evidence does not.

Documentation describes intended use, data, behavior, limitations, and ownership. It supports understanding, should be versioned, can become stale, and does not enforce policy. A governance process assigns decision rights, review triggers, exceptions, and lifecycle controls. It controls change and use, needs operational integration, includes recourse and expiry, and can stop or condition a release. Audit tests evidence and control operation against a defined criterion. It requires independence and scope, samples records and execution, finds gaps and exceptions, and does not replace day-to-day ownership.

New York City legislated that third column, and then somebody went and counted. Local Law 144 of 2021 requires an annual independent bias audit of automated employment decision tools, public availability of the audit information, and a notice to candidates at least 10 business days before use. The city's Department of Consumer and Worker Protection set the start date: “DCWP will begin enforcement of this law and rule on July 5, 2023.”

Then researchers went and checked what employers had actually published. Their 2024 study reports: “In this study, 155 student investigators recorded 391 employers' compliance with LL 144 and the user experience for prospective job applicants. Among these employers, 18 posted audit reports and 13 posted transparency notices.” Nearly all of the audits that were published reported an impact factor above 0.8.

An audit obligation and audit evidence are two different objects. One had been law since 5 July 2023. The other was 18 documents out of 391.

The controls should remain connected from collection to retirement. Every failure above happened at a seam between two stages, not inside one.

Collect and justify: record purpose, source, legal basis, consent where relevant, and minimization. That is the stage where Everalbum's express-consent question was decided. Access and transform: apply least privilege, segregation, secure environments, and transformation records. Train and evaluate: control datasets, exports, reviewer access, leakage, and sensitive slices. That is the stage where Ring's “review and annotation of Pre-March 2018 Covered Recordings” lives, and where derivation lineage is either recorded or lost for good. Serve and monitor: limit inputs, logs, telemetry, decisions, and downstream sharing — the served model is itself an extraction surface. Retain, delete, and retire: enforce schedules, deletion propagation, evidence retention, and closure. That is the stage that has to satisfy the 30- and 90-day clocks when they arrive.

Approval for one population and decision does not automatically authorize another. Model reuse, new data sources, threshold changes, larger scale, and new downstream actions can change risk even when weights remain unchanged.

The EDPB wrote that shape into its own test: whether a model is anonymous “should be assessed, based on specific criteria, on a case-by-case basis”. Anonymity is not a status a model earns once at launch and keeps. Ring's standard is operational in the same way. The deletion duty attaches to what is “identified or reasonably identifiable by the Defendant”, which makes the quality of your own asset map part of your legal exposure. A team that never traced its derivations has not reduced its obligations. It has only made them harder to discharge on oath, inside 90 days, over the signature of its Principal Executive Officer.

A governance system should define review triggers, decision rights, exceptions, expiry, evidence, and recourse. Static documents are not enough when the operational system keeps changing.

Convert policy language into a control with an owner, a trigger, evidence, and a failure response. State the obligation: purpose, population, data, action, and authoritative requirement. Map affected assets: sources, derivatives, models, logs, exports, and downstream consumers. Implement the control: access, minimization, review, retention, deletion, or recourse mechanisms. Produce bounded evidence: execution, exceptions, approvals, and effective times, recorded securely. Test failure and expiry: denial, deletion propagation, review triggers, and retirement.

Step three is where teams reach for machine unlearning, and the published numbers deserve reading before the roadmap is written. Bourtoule and colleagues measured their SISA framework against retraining a model from scratch, and published the result in 2021. The abstract: “Under no distributional assumptions, for simple learning tasks, we observe that SISA training improves time to unlearn points from the Purchase dataset by 4.63x, and 2.45x for the SVHN dataset, over retraining from scratch.” On ImageNet the figure is 1.36x, with accuracy degradation. A single-digit speed-up against full retraining is a real research result and a poor foundation for a 30-day deletion promise.

The regulator has looked at the same evidence. The ICO's outcomes report of 12 December 2024 records that “We received numerous arguments about machine unlearning. These mainly pointed to its theoretical application and not any current practical usage.”, and concludes: “Organisations must therefore have mechanisms in place to fulfil information rights requests for both the training data and, if a model contains personal data, the trained model itself.” Step five therefore tests the mechanisms that actually exist today — retrain-and-replace, isolation, and the documented model-handling policy — rather than the one the roadmap hopes for.

A policy that works only during the launch review will fail when an engineer adds a source, a team reuses an embedding, or an incident extends log retention. Integrate controls with registries, pipelines, access systems, and change review.

Measure exceptions, overdue actions, unauthorized use attempts, deletion completion, and appeal outcomes — not only document completion. New York City is the argument for that sentence. The bias-audit duty had been enforceable since 5 July 2023. A count of 391 employers turned up 18 audit reports and 13 transparency notices. What the law required was a document. What the study measured was whether anyone could find one. That gap is the whole difference between a governance programme and a governance claim.