Noise-aware Client Selection for carbon-efficient Federated Learning via Gradient Norm Thresholding

AI summary

Overview

• Research area: Sustainable AI / carbon-aware Federated Learning (FL), specifically client selection under unknown local data quality.
• Technical level: Intermediate (assumes familiarity with federated learning, gradient-based optimization, and basic carbon-intensity concepts).
• Scope: The paper proposes a modular, noise-aware extension to existing carbon-efficient client selection strategies, using gradient norm thresholding during probing rounds and utility-aware carbon budget allocation, evaluated on CIFAR-10, CIFAR-100, and Tiny ImageNet across 30 simulated US-region clients.

What This Paper Is About

Federated Learning can reduce AI's carbon footprint by training across geospatially distributed data centers that use locally available renewable energy, and client selection strategies decide which clients participate in each round. The problem is that FL is privacy-preserving, so the server cannot see whether a client's local data is high-quality or corrupted, yet existing strategies often pick clients with high training loss — which may indicate useful hard examples or noisy data. The goal is a modular mechanism that filters noisy clients using gradient norm statistics and simultaneously respects carbon budgets, so that model accuracy and sustainability are not traded off against each other unnecessarily.

Key Contributions

1. A modular probing-round extension that sits on top of state-of-the-art client selection strategies. At the start of federated training, all clients are evaluated once, and a gradient-norm-based "probing utility" replaces the usual loss approximation as the utility signal.
2. A gradient norm thresholding mechanism that retains only clients whose probing utility satisfies a variance-based coefficient threshold, excluding likely noisy clients from subsequent training rounds while remaining compatible with other selection methods and other utility metrics.
3. A utility-aware carbon budget allocation formulation adapted from Oort's reward mechanism, which selects the subset of clients maximizing total utility subject to a per-round emissions ceiling and a cap on the number of clients selected.
4. An empirical demonstration that combining the two produces a method (referred to as OortCAWT) that is robust to corrupted client data while spending limited carbon budgets on clean, high-utility clients.

Main Findings

• Loss-based selection favors noisy clients: Without thresholding, corrupted clients are frequently selected. Oort consistently prioritizes them because its utility metric is based on local training loss, which is typically inflated for noisy data; Oort's optional blacklisting only helps after a client has participated a specified number of rounds.
• Thresholding speeds and stabilizes convergence: In a setup where 6 of 30 clients hold noisy data, the thresholded variants (RandomWT and OortWT) converged faster and more stably and reached higher final accuracy than Random and Oort alone. A coefficient of c = 0.5 gave a good balance in this setup.
• Noisy clients can be the dirtier choice for emissions too: The corrupted clients (IDs 0–5 in the paper's carbon intensity figure) had a lower average carbon intensity than the others, so selecting them increased carbon emissions while achieving lower accuracy.
• Probing pays for itself: Although the probing round introduces a one-time computational cost, it allows maximum accuracy to be reached earlier, enabling a reduction in training rounds; the paper reports emissions measured at the point of maximum accuracy.
• Carbon budgets can preserve accuracy: With clean data, the utility-aware budget allocation (OortCA) achieved final accuracy comparable to the unconstrained Oort baseline while using only 40% of its emissions. Budgets were tested starting from zero carbon (curtailment-only clients) and increased in 10% increments.
• Noise and budgeting interact: Under noisy data, model performance varied drastically, but filtering noisy clients via gradient norm improved robustness; combined with carbon-aware budgeting, it enabled accuracy improvements while accounting for reduced carbon emissions.
• Results generalize across models and datasets: Figure 8 reports DenseNet-121 evaluated on CIFAR-100 and EfficientNet-B1 evaluated on Tiny ImageNet, again with noise added to 6 out of 30 clients, showing that with clean data a constrained client pool can achieve similar model performance, while noisy conditions require filtering for robustness.

Methodology in Plain English

The authors simulate a federated system of 30 clients, mapped to 30 US energy regions, and feed each one real hourly carbon intensity data from Electricity Maps, covering 100 training rounds of one hour each (2023-01-15 00:00:00 to 2023-01-19 04:00:00 UTC). Each client is assumed to consume 1 kWh per training round, and if energy curtailment data indicates available curtailed energy, that client's intensity is set to 0.

To create a data-quality problem, they train on CIFAR-10 with a non-IID Dirichlet distribution (α = 10). A simple CNN with 2 convolutional and 3 fully connected layers is trained with 10 clients per round, 2 local epochs, batch size 32, and Adam at a learning rate of 0.001. For the noisy scenario, the data of 6 clients is replaced with corrupted variants made by adding zero-mean Gaussian noise with standard deviation σ = 1 to each image and clipping pixel values to [0, 1].

The noise-aware part works in two steps. First, a one-time probing round computes a utility for every client from the L2 norm of the gradients of its local samples, rather than from its training loss — the idea being that this reflects the curvature of the loss landscape and better distinguishes informative data from corrupt data. Second, the server applies a threshold: a client stays only if its probing utility is at least c times the maximum utility, where c is a configurable coefficient between 0 and 1.

For the carbon-aware part, the client selection is framed as an optimization: pick the subset of clients with the highest total utility such that the round's total carbon stays within a budget and the number of selected clients does not exceed K. Unused budget carries over between rounds, and if the budget runs out before enough clients are chosen, the remaining slots go to curtailment clients with the highest utility scores. This is compared against original Oort with all clients available, whose total emissions serve as the baseline against which budget levels are defined.

Why This Matters

This work matters because it exposes a hidden conflict in carbon-aware federated learning: strategies that chase renewable energy availability and high training loss can systematically recruit the clients with the worst data, harming accuracy while also burning carbon. By showing that a single gradient-norm probing round can identify and exclude those clients, the paper offers a practical, modular fix that does not require breaking privacy or redesigning the selection stack.

Real-world applications:

• Scheduling training across geospatially distributed data centers so workloads follow renewable availability without degrading model quality.
• Edge and on-device federated deployments where local data quality cannot be audited and corrupted updates must be filtered out.
• Carbon-budgeted ML operations, where organizations cap emissions per training period and need to spend that budget on the most useful clients.
• Onboarding new clients into an existing federated system, where a one-time probe can reveal whether their data is worth including.

Industry relevance: Any organization running distributed or federated training under emissions reporting constraints, particularly cloud providers operating data centers in multiple energy regions, could adopt the probing-round and budget-allocation mechanisms independently of the rest of their pipeline, since the approach is explicitly designed as a modular layer on top of existing selection strategies.

Future Directions

• Data valuation for carbon-aware FL: Evaluate techniques such as Federated Shapley Values and Gradient Alignment in carbon-aware settings to balance fairness, bias mitigation, and carbon efficiency.
• Asynchronous Federated Learning: Remove reliance on potentially inaccurate and costly carbon intensity forecasts by coupling asynchronous training with data valuation and carbon budgeting.
• Aligning with critical learning periods: Match the training phases where models are most sensitive to high-quality data with moderately carbon-intensive, high-value data to improve both convergence and sustainability.
• Cheaper probing: Estimate data coresets via a single round of local inference to lower the cost of gradient norm calculations and ease the onboarding of new clients.
• Open question on threshold selection: The paper notes that choosing c in practice would require evaluation on a small, representative subset of the data, which remains a practical design question.

Target Audience

Researchers and practitioners working on federated learning, sustainable/carbon-aware machine learning, and distributed training systems; engineers designing client selection or participation policies for multi-region data centers; and graduate students interested in the intersection of AI efficiency, energy systems, and privacy-preserving training. Readers without a background in federated learning or gradient-based optimization will find the carbon-budget and probing concepts accessible, but the utility and thresholding formulations require intermediate familiarity.

Authors’ abstract

Training large-scale Neural Networks requires substantial computational power and energy. Federated Learning enables distributed model training across geospatially distributed data centers, leveraging renewable energy sources to reduce the carbon footprint of AI training. Various client selection strategies have been developed to align the volatility of renewable energy with stable and fair model training in a federated system. However, due to the privacy-preserving nature of Federated Learning, the quality of data on client devices remains unknown, posing challenges for effective model training. In this paper, we introduce a modular approach on top to state-of-the-art client selection strategies for carbon-efficient Federated Learning. Our method enhances robustness by incorporating a noisy client data filtering, improving both model performance and sustainability in scenarios with unknown data quality. Additionally, we explore the impact of carbon budgets on model convergence, balancing efficiency and sustainability. Through extensive evaluations, we demonstrate that modern client selection strategies based on local client loss tend to select clients with noisy data, ultimately degrading model performance. To address this, we propose a gradient norm thresholding mechanism using probing rounds for more effective client selection and noise detection, contributing to the practical deployment of carbon-efficient Federated Learning.