Not All Retained Clients Are Equal: Client Selection for Federated Unlearning

AI summary

Overview

Research area: Federated learning and machine unlearning, specifically client-level federated unlearning (FU) — removing a designated client's influence from a trained global model without full retraining.

Technical level: Advanced. The core idea is intuitive, but the paper formalizes it with cosine-similarity alignment scores, an online critical-learning-period detector, and a memory/compute complexity analysis.

Scope: The paper proposes PDF, a client-selection framework that ranks retained clients by their path-dependent training history so that only the top-k of them need to participate in retraining-based unlearning.

What This Paper Is About

When a client must be removed from a federated model (for legal or security reasons), existing methods either retrain with every remaining client — expensive and impractical when clients come and go — or manipulate model parameters in ways that trade forgetting quality against the accuracy of the retained clients. The authors argue that retained clients are not equally useful for unlearning: a client's influence depends on when it contributed during training (especially during critical learning periods, CLPs) and how strongly its updates became coupled with the target client's updates. The goal is to exploit this training-time history to pick a small subset of retained clients that both preserves retained knowledge and avoids re-injecting the target's influence.

Key Contributions

1. A path-dependent framing of client-level FU. The paper argues that client influence is path-dependent — shaped by when a client participated (particularly during CLPs) and by its historical dependence on the target clients — rather than a stateless, post-hoc property of the final model.

2. The PDF framework. A CLP-aware client-selection method that maintains lightweight, target-agnostic training histories (one accumulated classification-layer update direction and two scalars per client) and, at request time, combines them with a request-conditioned target-coherence score to rank retained clients.

3. Retraining-based unlearning with only the top-k clients. PDF performs unlearning using the k clients that are globally informative yet weakly coupled to the target set, reducing participation, communication, and computation while balancing forgetting against retained-data generalization.

4. Empirical validation across three datasets and two data distributions. Experiments on FMNIST, CIFAR-10, and CIFAR-100 under IID and non-IID partitions, plus ablations isolating the ranking signal and the CLP weighting.

Main Findings

• Strong forgetting on non-IID CIFAR-10: PDF reaches 0% forgetting accuracy (FA) while keeping MIA AUC close to Retrain (54.76% versus 55.64%, a −0.88 point gap), i.e., retraining-like privacy behavior. The paper's headline claim is 0% FA with retraining-like MIA AUC on non-IID CIFAR-10.

• Best retained accuracy on CIFAR-100: All three PDF variants beat the strongest competing baseline in retained accuracy (RA) under both distributions — up to 49.13% versus 42.22% at α = 0.1 and 54.38% versus 47.50% at α = 1.0, a maximum gain of 6.91 percentage points.

• Competitive on FMNIST: PDF reaches 86.89% RA at α = 0.1 (Retrain: 88.20%), and the forgetting-oriented variant reaches 0% FA.

• Exceeding Retrain on heterogeneous CIFAR-10: Under α = 0.1, PDF with β = 0.9 attains 57.10% RA, above Retrain (55.28%) and FOUL (52.10%); PDF with β = 0.5 combines 54.48% RA with 0% FA.

• β controls the forgetting–utility trade-off as designed: Increasing β from 0.1 to 0.9 raises RA from 82.60% to 86.89% on FMNIST and from 45.22% to 57.10% on CIFAR-10 at α = 0.1. Variation is weaker on CIFAR-100, suggesting lower sensitivity to β there.

• Efficiency gains relative to Retrain: Approximately 69% less communication and 57–61% less computation, measured over training plus unlearning combined (e.g., CIFAR-10 communication 845.48 GB → 259.46 GB; FMNIST 15.62 GB → 4.79 GB).

• Compact storage footprint: The learning-stage cache costs O(N·d_cls) space, independent of the number of training rounds, versus O(P + |I_ret|·P) for Retrain, O(T_learn·M·P + P) for Exact-Fun and FedRecovery, and O(N·P + P) for CONDA and FOUL.

• Diminishing returns from more selected clients: At β = 0.5 on FMNIST (α = 0.1), RA rises from 79.26% at k = 10 to 85.97% at k = 25, but only to 86.40% at k = 30, while communication and computation keep climbing. At β = 0.9, going from k = 20 to 25 adds 0.9 points of RA, versus 0.1 points from 25 to 30. MIA AUC stays within 0.41 points of Retrain across the whole sweep, motivating the default k = 25.

• The ranking signal itself matters: Against selection variants (non-CLP weighting, reversed ranking, high-coherence selection) at β = 0.5, k = 25, PDF is best on CIFAR-10 at α = 0.1 with 0.00 FA and 54.48% RA, versus 0.19/49.60 for non-CLP, 4.44/41.09 for the reversed ranking, and 7.65/37.65 for high-coherence selection.

Methodology in Plain English

Training stage (target-agnostic). The server runs standard FedAvg with N = 64 clients, sampling M = 16 per round, using local learning rate 0.01 and batch size 32. While training, it does two cheap things. First, it watches a proxy for how much the loss is dropping each round (the federated gradient norm) and flags a round as "critical" when that proxy jumps by at least τ_CLP = 0.01 relative to the previous round — this catches critical periods whenever they occur, not just early in training. Second, for each participating client it records how similar the client's classification-layer update was to the realized global update (a normalized cosine similarity), and accumulates that client's normalized update directions over time, with critical rounds up-weighted by a factor of 1 + λ (λ = 0.7).

What gets stored. Per client: one accumulated direction vector for the classification layer, plus two scalars (accumulated alignment and accumulated weight). Nothing about the target set is needed, no model snapshots are kept, and the cost does not grow with the number of rounds.

Unlearning stage (request-conditioned). When a target set is named, the server averages the target clients' historical directions into a single reference direction, then scores each retained client as β × (historical global alignment) + (1 − β) × (1 − target coherence), where target coherence is the cosine similarity between that client's historical direction and the target's. High β favors clients that consistently followed the global trajectory (good for retained utility); low β favors clients historically decoupled from the target (good for forgetting). The top-k scoring clients (k = 25 by default) then run T_unlearn = 20 rounds of retraining-based unlearning.

Evaluation. The unlearning request is triggered by simulating a Byzantine label-flipping attack from a single client. Baselines are Retrain, CONDA, FedRecovery, FOUL, FedEraser, Exact-Fun, FUSED, and GDFA. Models are a CNN for FMNIST and ResNet-18 for CIFAR-10 and CIFAR-100. Metrics are forgetting accuracy, retained accuracy, MIA AUC gap to Retrain, communication in GB, computation in PFLOPs, and space complexity; computation and communication are counted over both training and unlearning because several baselines alter the training protocol.

Why This Matters

Impact on research. The paper reframes federated unlearning from a model-manipulation problem into a client-selection problem, and connects the CLP literature (previously used only to improve training) to influence removal. It also shows that unlearning does not require all retained clients, challenging the prevailing assumption in retraining-based FU that more participants means better fidelity.

Real-world applications.

• GDPR-style erasure requests in cross-institution federated systems such as hospitals, where a partner must be removed after the fact.
• Malicious or corrupted client eviction, as simulated here with label-flipping Byzantine attacks.
• Client churn in cross-device federated learning, where many retained clients are simply offline when an unlearning request arrives, making all-client retraining infeasible.
• Audit and compliance pipelines that need retraining-like behavior (as measured by MIA AUC close to Retrain) rather than approximate parameter surgery.

Industry relevance. A ~69% communication reduction and 57–61% computation reduction relative to Retrain, combined with O(N·d_cls) server storage that does not grow with training rounds, matters for operators paying for bandwidth and compute. Because the training-time bookkeeping is target-agnostic, an operator can deploy it before knowing which clients will ever need removal.

Future Directions

• Validating beyond the label-flipping, single-client scenario. The paper simulates one Byzantine client via label flipping; multi-client target sets of varying size, and other request types (sample- or class-level), are not evaluated here. The truncated content does not report such experiments.

• Reducing sensitivity to β and to CLP detection settings. RA swings substantially with β on FMNIST and CIFAR-10 (82.60% to 86.89%, and 45.22% to 57.10%), while CIFAR-100 is comparatively flat. A principled way to set β and τ_CLP per deployment, rather than selecting them empirically, is an open question.

• Extending the history signal beyond the classification layer. All statistics are computed on the classification layer for storage and compute reasons; whether richer layer-wise histories improve selection at acceptable cost is untested.

• Guaranteeing the formal tolerances. The problem statement defines retained-utility and forgetting tolerances (δ and ε_f) relative to a retraining reference, but the paper's evidence is empirical accuracy and MIA measurements across three datasets; a theoretical characterization of when top-k selection satisfies those bounds is not provided.

Target Audience

Researchers and graduate students working on federated learning, machine unlearning, or privacy-preserving model maintenance; engineers designing federated systems that must support erasure requests under intermittent client availability; and practitioners evaluating whether full retraining is truly necessary after a client is removed. Readers should be comfortable with FedAvg, cosine similarity, and standard unlearning evaluation metrics such as forgetting accuracy and membership inference AUC.

Authors’ abstract

Federated unlearning (FU) aims to remove a designated target client's influence from a trained model without the prohibitive cost of retraining from scratch. Existing methods largely treat unlearning as a post-hoc, stateless operation, overlooking that client influence is path-dependent: contributions made during training, particularly during critical learning periods (CLPs), can have disproportionate and persistent effects. Moreover, knowledge overlap and intermittent client availability make relying on all retained clients inefficient and impractical. We introduce PDF, a path-dependent client-selection framework based on the insight that retained clients are not equally valuable for unlearning. During training, PDF maintains lightweight, target-agnostic, CLP-weighted histories summarizing each client's accumulated update direction and global-model alignment. Upon an unlearning request, these histories identify and rank retained clients that are globally informative yet weakly coupled to the target, enabling retraining-based unlearning with only the top-$k$ clients over a small number of rounds. Experiments on FMNIST, CIFAR-10, and CIFAR-100 under IID and non-IID settings show that PDF achieves effective forgetting and strong retained utility with substantially fewer retained clients. In particular, PDF achieves $0\%$ forgetting accuracy with retraining-like MIA AUC on non-IID CIFAR-10, improves retained accuracy by up to $6.91$ percentage points over the strongest competing FU baseline on CIFAR-100, and reduces communication by approximately $69\%$ and computation by $57$--$61\%$ relative to Retrain.