HealSplit: Towards Self-Healing through Adversarial Distillation in Split Federated Learning

AI summary

Overview

• Research area: Privacy-preserving distributed machine learning, specifically defenses against data poisoning attacks in Split Federated Learning (SFL).
• Technical level: Advanced. The paper combines graph-based anomaly detection, generative adversarial networks, multi-teacher knowledge distillation, and a gradient-variance theory argument.
• Scope (1 sentence): The paper proposes and evaluates HealSplit, a unified end-to-end defense framework that detects and repairs poisoned smashed data in SFL across five categories of poisoning attacks and four benchmark datasets.

What This Paper Is About

Split Federated Learning lets clients compute only part of a model locally and send intermediate representations ("smashed data") to a server, which is meant to keep data private and reduce client computation. However, attackers can poison labels, input data, smashed data, or model weights, and existing defenses were adapted from standard Federated Learning, where they can see complete model updates — an assumption that does not hold in SFL. The goal of this paper is to build a single defense system that can both detect poisoned samples and replace them with semantically valid substitutes, without knowing in advance which attack type is being used.

Key Contributions

1. First unified SFL defense framework. HealSplit targets five attack types simultaneously — label poisoning, data poisoning, smashed data poisoning, weight poisoning, and multi-vector poisoning — rather than a single isolated threat.
2. Topology-aware detection via Topological Anomaly Score (TAS). The method builds a KNN graph over smashed data and runs Personalized PageRank (PPR) on it to score nodes that form locally dense but globally isolated clusters, using a kernel-density-estimation-based adaptive threshold.
3. Generative recovery with a consistency validation student. A GAN generates substitute smashed data for detected anomalies, and a student model filters those substitutes for semantic and label consistency.
4. Adversarial multi-teacher distillation with an Anomaly-Influence Debiasing (AD) Teacher. The student is trained with supervision from a clean Vanilla Teacher and an anomaly-aware AD Teacher, combined with a Gradient Interaction Score (GIS) matrix and momentum-adaptive weighting.
5. Theoretical grounding. The paper defines Inter-Server Gradient Variance (SGV) and proves in Theorem 1 that under a stated clean-sample-ratio condition, HealSplit's robust objective bounds gradient dissimilarity by the original SGV.

Main Findings

• High accuracy under every tested attack: HealSplit reports 97.17±1.27 (No Attack), 96.86±0.77 (DP), 95.99±1.69 (WP), 96.75±0.86 (SP), 96.72±0.64 (LP), 93.88±0.60 (DP+SP), 92.44±0.73 (WP+SP), and 93.88±1.38 (LP+SP) on MNIST, maintaining over 92% accuracy across all attack scenarios.
• Strong margins over advanced baselines on composite attacks: FLTrust drops to 11.33±4.89 under WP+SP, while HealSplit reports 92.44±0.73. On the simple DP attack, FedAvg falls to 10.12±0.85 while HealSplit reports 96.86±0.77.
• Ablation shows the AD Teacher and distillation matter most: Removing the AD Teacher lowers accuracy from 93.88±0.10 to 87.34±0.16 on MNIST; removing distillation lowers it to 74.38±3.20. Removing the Vanilla Teacher gives 90.99±1.16, and removing the adversarial mechanism gives 92.74±1.68. The same ordering appears on F-MNIST (84.11±0.47 baseline), CIFAR-10 (53.87±1.11 baseline), and HAM10k (72.27±0.52 baseline).
• Resilience across system configurations: Under varying client counts and rising malicious-client proportions, HealSplit maintains stable accuracy while DnC degrades with increased variance. The text does not report specific numeric values for these curves.
• Generalization across data distributions and architectures: HealSplit stays above 85% accuracy on MNIST as it becomes increasingly non-IID, while baselines degrade. It also outperforms DnC across ResNet-18, ResNet-152, and VGG16, and on the real-world HAM10000 distribution shift.
• Partial degradation under adaptive attacks: When an attacker minimizes the TAS divergence between poisoned and clean smashed data, HealSplit's performance drops noticeably but it still outperforms the strongest existing defenses across multiple datasets. Specific accuracy values for the adaptive attack are not reported in the text.

Methodology in Plain English

The researchers focus on the one component that every SFL attack must pass through: the smashed data sent from client to server.

1. Detect. They treat each smashed-data sample as a node in a graph, connecting nodes that are close to each other in feature space. Poisoned samples tend to cluster tightly together while sitting far from the legitimate data. A PageRank-style score is computed for each node, and a threshold derived from the score distribution — using kernel density estimation — flags the outliers as poisoned. This threshold adapts automatically, so no manual tuning of a neighbor count or clipping value is required.

2. Repair. A GAN trained on the smashed data identified as clean learns to generate replacements. Because the GAN may produce outputs that look plausible but do not fit the correct label, a student model acts as a gatekeeper and only accepts generated samples with strong confidence and label agreement.

3. Train the gatekeeper. The student is taught by two teachers in an adversarial setup. One teacher, trained on clean data, preserves ordinary semantics. The other, the AD Teacher, learns to recognize the fingerprints of poisoning behavior by examining how gradients for three related tasks — poisoning identification, client identification, and category classification — align with each other. These alignment scores are fused with the topological scores into an influence matrix that decides which label-to-label information is allowed to propagate. A momentum-based rule automatically rebalances how much the student listens to each teacher, preventing either from dominating training.

4. Validate. Experiments cover four datasets, ten baseline defenses, seven attack configurations, different client counts, different malicious-client proportions, IID and non-IID splits, and three model architectures.

Why This Matters

• Research impact: The paper argues that defenses transplanted from Federated Learning are structurally mismatched to SFL, because SFL never exposes complete model updates. It offers a defense organized around the actual SFL data flow (smashed data) plus a theoretical variance bound, giving the SFL security community a new baseline to build on and a target to attack.
• Real-world applications:
  • Mobile and edge health analytics, where patient images such as dermatology scans (HAM10000 in this paper) must be classified without centralizing raw data.
  • Cross-institution medical imaging consortia, where a malicious participant could otherwise corrupt a shared diagnostic model.
  • Smartphone or IoT keyboard and vision models trained collaboratively across many devices with limited compute.
  • Regulated sectors such as finance and insurance that need multi-party model training with auditable resistance to tampering.
• Industry relevance: Any deployment of split learning or federated learning where participants are not fully trusted — including cross-silo collaborations between competitors or between hospitals — needs a mechanism that works without seeing raw client data or complete gradients. HealSplit's attack-agnostic, self-tuning design lowers the operational burden compared to methods requiring manual thresholds such as Krum's neighbor count or SparseFed's norm clipping parameter.

Future Directions

• Hardening against adaptive attackers. The paper shows a measurable drop when attackers deliberately flatten TAS divergence, leaving open how to detect adversaries who optimize directly against the detection score.
• Determining the limits of the theoretical guarantee. Theorem 1 relies on a specific clean-to-poisoned sample ratio condition; its behavior when that condition is violated, or when the malicious client fraction α grows large, is not characterized in the text.
• Scaling to larger and more heterogeneous deployments. The reported configuration uses 10 clients, 20% malicious, and 100 epochs; whether the approach holds at much larger scale and under more severe non-IID skew is left open.
• Reducing overhead. Training a GAN, two teachers, and a student alongside the main SFL model incurs additional computation and communication costs that the paper does not quantify.

Target Audience

Researchers and practitioners working on federated and split learning security, trustworthy distributed machine learning, and robust aggregation. It is also relevant to engineers deploying privacy-preserving training across untrusted participants, and to graduate students studying adversarial machine learning who want a concrete example of integrating graph-based detection, generative repair, and knowledge distillation into one system. Readers should be comfortable with gradient-based optimization, adversarial training concepts, and basic graph propagation ideas.

Authors’ abstract

Split Federated Learning (SFL) is an emerging paradigm for privacy-preserving distributed learning. However, it remains vulnerable to sophisticated data poisoning attacks targeting local features, labels, smashed data, and model weights. Existing defenses, primarily adapted from traditional Federated Learning (FL), are less effective under SFL due to limited access to complete model updates. This paper presents HealSplit, the first unified defense framework tailored for SFL, offering end-to-end detection and recovery against five sophisticated types of poisoning attacks. HealSplit comprises three key components: (1) a topology-aware detection module that constructs graphs over smashed data to identify poisoned samples via topological anomaly scoring (TAS); (2) a generative recovery pipeline that synthesizes semantically consistent substitutes for detected anomalies, validated by a consistency validation student; and (3) an adversarial multi-teacher distillation framework trains the student using semantic supervision from a Vanilla Teacher and anomaly-aware signals from an Anomaly-Influence Debiasing (AD) Teacher, guided by the alignment between topological and gradient-based interaction matrices. Extensive experiments on four benchmark datasets demonstrate that HealSplit consistently outperforms ten state-of-the-art defenses, achieving superior robustness and defense effectiveness across diverse attack scenarios.