WARP: Weight Teleportation for Attack-Resilient Unlearning Protocols

AI summary

Overview

Research area: Machine learning privacy — specifically approximate machine unlearning and the adversarial attacks and defenses that surround it.

Technical level: Advanced. The paper assumes familiarity with gradient-based optimization, membership inference and reconstruction attacks, singular value decomposition (SVD), orthogonal projections, and neural network symmetry transformations.

Scope: The paper demonstrates that approximate unlearning leaks information about the data it claims to delete, proposes tailored attacks that exploit this leakage, and introduces a plug-and-play symmetry-based defense (WARP) that reduces attack success across six unlearning algorithms, three datasets, and both black-box and white-box threat models.

What This Paper Is About

Machine unlearning tries to remove the influence of specific training examples (a "forget set") from an already-trained model without the cost of retraining from scratch. The paper shows that this process can itself leak the very data it is meant to erase: an attacker who sees both the original model and the unlearned model can compare them and infer or reconstruct what was forgotten. The goal is to characterize why this leakage happens and to build a defense that works on top of existing unlearning methods without hurting their accuracy on retained data.

Key Contributions

1. Unlearning-specific privacy attacks. The authors design a black-box membership inference attack (U-LiRA, an adaptation of LiRA to unlearning) and a white-box gradient-difference membership inference test, plus a white-box reconstruction attack that uses orthogonal subspace filtering to isolate the forget-set gradient from the retain-set gradient inside the parameter change Δθ. These attacks show that several state-of-the-art unlearning methods (including NGP and SCRUB) remain vulnerable.

2. WARP, a symmetry-based, plug-and-play defense. WARP applies "teleportation" steps — transformations that move model parameters while leaving predictions unchanged — to reduce forget-set gradient norms and to increase the displacement of the unlearned parameters from the original ones. It requires no training-time per-sample statistics and can be inserted into gradient-based post-hoc unlearning algorithms.

3. A general symmetry framework. The defense is formulated over an abstract prediction-preserving symmetry map T_φ, with the primary instantiation using retain null-space projection and a second instantiation using change-of-basis teleportation described in the appendix. The authors also develop teleportation-aware information-theoretic bounds on gradient-based reconstruction in Appendix O.

4. Comprehensive evaluation. Six unlearning algorithms (NegGrad+/NGP, SCRUB, SalUn, PGU, BadTeacher/BT, and SRF-ON/SF) are evaluated on CIFAR-10, Tiny-ImageNet, and ImageNet-1K with ResNet-18 and ViT-B/16, under both black-box and white-box settings, including a comparison against projected DP–Langevin unlearning (Appendix M) and runtime overhead analysis (Appendix J).

Main Findings

• Two drivers of leakage. The paper attributes privacy leakage to (i) large gradient norms of forget-set samples in the original model, and (ii) the close proximity of unlearned parameters to the original parameters, which lets an attacker read the forget-set gradient out of the parameter difference.

• Black-box vulnerability and defense gains. Under U-LiRA with T=64 shadow models (10 random forget sets each), WARP reduces adversarial advantage by up to 64.4 percent (NGP, on the full forget set). For NGP, TPR@1 on all samples nearly halves from 0.030 to 0.014; SCRUB's most-memorized-slice AUC drops from 0.710 to 0.610; SF's AUC falls to 0.501, near random.

• White-box vulnerability is more severe. The white-box Gaussian gradient-difference test (640 unlearned models, m=1000 non-members, ridge λ=10⁻³, top-10 percent most-variant coordinates) shows the largest gains for PGU, whose AUC drops from 0.659 to 0.533 (92.9 percent improvement), and for BT, which drops from 0.938 to 0.907 (49.2 percent). Some methods that look robust in black-box evaluation still leak heavily when gradients are exposed.

• A narrow corner case. For SCRUB, teleportation lowers the ROC above 10⁻³ FPR but slightly raises TPR below 10⁻³ FPR, attributed to knowledge distillation interacting with symmetry moves that amplify high-leverage directions.

• Reconstruction is disrupted. On ImageNet-1K with ResNet-18 and NGP (100 sampled points, retain minibatch size 5, m=100 probe samples, rank k preserving 90 percent gradient energy), teleportation degrades reconstruction: PSNR falls from 10.74 ± 0.31 dB to 7.38 ± 0.40 dB (+45.5 percent defense improvement), LPIPS(VGG) rises from 0.56 to 0.68 (+21.2 percent), LPIPS(Alex) from 0.34 to 0.46 (+26.1 percent), SSIM drops from 0.12 to 0.08 (+31.6 percent), test MSE rises from 0.10 to 0.21 (+52.4 percent), and feature MSE from 5.39 to 11.28 (+52.2 percent).

• Utility is largely preserved. Adding teleportation keeps retain-set accuracy stable across methods, with the only noticeable drop being NGP (about one percentage point). For BT and SF, teleportation even improves test accuracy.

• No single algorithm dominates. The radar-chart comparison shows that algorithms strong on one axis (for example SF in black-box auditing) can be weak on another (white-box auditing or test accuracy), motivating evaluation under both threat models.

Methodology in Plain English

Attacks first. The authors build two membership inference tests and one reconstruction attack specifically tailored to the unlearning setting, where the adversary holds both the pre- and post-unlearning models. The reconstruction attack is the cleverest part: because the parameter change Δθ mixes retain-set and forget-set gradients, naively inverting it fails. So the authors estimate the dominant gradient subspaces of the original and unlearned models using SVDs of gradients on a small probe set, then project the update onto the original model's subspace and orthogonal to the unlearned model's subspace. This leaves a high signal-to-noise estimate of the forget gradient, which is then inverted to reconstruct the forgotten image.

Defense second. WARP exploits the fact that deep networks have symmetries — transformations of the weights that leave the model's predictions unchanged. A "teleportation" step moves the parameters along such a symmetry while deliberately choosing the direction that shrinks forget-set gradient norms and pushes the parameters away from their original values. In the primary instantiation, this is done by projecting the teleportation update per layer onto the complement of the subspace spanned by retain-set activations (keeping the top-k left singular vectors that capture typically 95–99 percent of retain variance). Teleportation steps are interleaved with the standard unlearning update every S steps, so the whole defense slots into existing pipelines without needing training-time per-sample gradients or stored statistics. Because the attacker must now disentangle the forget signal from symmetry-induced noise, both membership inference and reconstruction become much harder.

Why This Matters

Impact on research. The paper reframes unlearning privacy risk around a measurable quantity — forget-set gradient norm — and ties it to neural network symmetry, an underexplored optimization principle. It also argues that black-box auditing alone is insufficient and that white-box auditing with gradient- or weight-based evidence should become standard practice in the unlearning literature.

Real-world applications:

• Regulatory deletion requests (for example, "right to be forgotten" or GDPR-style erasure) where a service must prove that a user's data no longer influences a deployed model.
• Machine-learning-as-a-service providers that release or update model weights and need to avoid leaking deleted user data through version differences.
• Sensitive-domain models in healthcare or finance, where reconstruction of a forgotten record would be a serious breach.
• Third-party model auditors who need realistic attack tools to test whether a vendor's unlearning claim actually holds up.

Industry relevance. Because WARP is plug-and-play and does not require training-time statistics, it can be layered onto existing approximate unlearning pipelines rather than replacing them. The reported runtime overhead is analyzed separately in Appendix J, and the authors report negligible cost for the teleportation steps, which matters for providers that must unlearn repeatedly at scale.

Future Directions

1. Extend Langevin-based privacy analyses to practical unlearning that combines gradient ascent with symmetry moves, to give formal grounding to the empirical privacy gains.

2. Counter low-rank weight signals. Recent work shows approximate unlearning leaves low-rank weight signals that can be reversed by re-unlearning or removed by quantization; the authors suggest exploring teleportation applied directly to weights to obscure these signals and resist reversal.

3. Adopt stronger or cheaper symmetry mechanisms. As the study of neural network symmetries advances, WARP can inherit new estimators and richer invariance families by simply swapping out its symmetry map.

4. Address the SCRUB corner case and other algorithm-specific interactions, such as those involving knowledge distillation, where symmetry moves were observed to occasionally amplify high-leverage directions at very low FPR.

Target Audience

Researchers working on machine unlearning, privacy-preserving machine learning, and trustworthy ML auditing; practitioners who deploy models with deletion guarantees; and security engineers who need to evaluate whether an unlearning procedure actually removes information rather than merely hiding it from black-box queries. The paper is most useful to readers already comfortable with gradient-based attacks and linear algebra, though the high-level motivation — that deleting data can leak it — is accessible to a broader audience.

Authors’ abstract

Approximate machine unlearning aims to efficiently remove the influence of specific data points from a trained model, offering a practical alternative to full retraining. However, it introduces privacy risks: an adversary with access to pre- and post-unlearning models can exploit their differences for membership inference or data reconstruction. We show these vulnerabilities arise from two factors: large gradient norms of forget-set samples and the close proximity of unlearned parameters to the original model. To demonstrate their severity, we propose unlearning-specific membership inference and reconstruction attacks, showing that several state-of-the-art methods (e.g., NGP, SCRUB) remain vulnerable. To mitigate this leakage, we introduce WARP, a plug-and-play teleportation defense that leverages neural network symmetries to reduce forget-set gradient energy and increase parameter dispersion while preserving predictions. This reparameterization obfuscates the signal of forgotten data, making it harder for attackers to distinguish forgotten samples from non-members or recover them via reconstruction. Across six unlearning algorithms, our approach achieves consistent privacy gains, reducing adversarial advantage (AUC) by up to 64% in black-box and 92% in white-box settings, while maintaining accuracy on retained data. These results highlight teleportation as a general tool for reducing attack success in approximate unlearning.