Contrastive Adversarial Training for Robust Graph Neural Networks under Label Poisoning

AI summary

Overview

Research area: Machine learning — robust graph representation learning, specifically defenses against adversarial label poisoning in semi-supervised node classification with Graph Neural Networks (GNNs).

Technical level: Advanced. The paper assumes familiarity with GNN message passing, bilevel (min–max) optimization, adversarial training, and supervised contrastive learning.

Scope: The paper proposes CoLAT, a contrastive adversarial training framework that uses a structure-aware contrastive detector to select high-risk nodes and flip their labels adversarially during training, and evaluates it against label-poisoning attacks on several graph benchmarks.

What This Paper Is About

GNNs are vulnerable to label poisoning: an attacker who corrupts a small number of training labels can degrade performance across the whole graph because message passing propagates the error. Most existing defenses were designed for random label noise, so they struggle to tell a deliberately poisoned node apart from a genuinely hard example. The paper's goal is a defense that uses graph structure itself — specifically, whether a node's label agrees with its neighborhood — to find and neutralize adversarially corrupted labels.

Key Contributions

1. CoLAT framework. A contrastive-adversarial training framework that models label poisoning explicitly as an adversarial process and approximates the intractable minimax formulation (Equation 1, budget k) with a tractable, structure-aware bilevel procedure.
2. Structurally-guided inconsistency scoring. A separate contrastive detection model g_φ (same architecture as the classifier f_θ but separate parameters) whose ℓ₂-normalized embeddings are used to score nodes by label–neighborhood disagreement. Contrastive learning is repurposed for node selection to generate adversarial examples, rather than as auxiliary representation regularization as in prior work.
3. Alternating optimization scheme. The inner maximization is decomposed into (i) selecting the top-k most inconsistent labeled nodes and (ii) assigning each selected node the loss-maximizing label, avoiding combinatorial search and enabling scalable adversarial training.
4. Empirical robustness study. Results across multiple benchmarks and attack types (LafAK, NTK, SMulti), attack budgets, architectures (GCN, GAT, GraphSAGE), a large graph (OGBN-ArXiv), and a heterophilic graph (Actor).

Main Findings

• Robust accuracy gains on homophilic benchmarks. On Cora ML, CoLAT reaches 0.6689 under 20% NTK poisoning and 0.5342 under 30% NTK, versus 0.5378 and 0.4626 for clean (undefended) training and 0.5163 / 0.4581 for PIGNN. Under SMulti 30%, CoLAT reaches 0.6739 versus 0.3561 for clean training.
• Largest margins at severe poisoning. The paper highlights 20% and 30% NTK poisoning on Cora ML and Citeseer as settings where baselines drop markedly while CoLAT holds up (Citeseer at 30% NTK: 0.4286 for CoLAT versus 0.3319 for clean training; at 30% SMulti: 0.5937 versus 0.4100).
• PubMed separation under strong attacks. At 30% NTK poisoning CoLAT reports 0.7636 versus 0.3520 for clean training; at 30% SMulti, 0.7114 versus 0.4989.
• Clean-data performance preserved on the main benchmarks. CoLAT's clean-dataset accuracies are 0.8377 (Cora ML), 0.7420 (Citeseer) and 0.8171 (PubMed), each at or above the corresponding vanilla GCN values of 0.8270, 0.7410 and 0.7940. On OGBN-ArXiv (0.7055 vs 0.7086) and Actor (0.2777 vs 0.2942) it is slightly below clean training.
• Label sanitization. Under SMulti 10% on Cora ML, CoLAT restores 36% of poisoned labels to their clean counterparts while also introducing additional adversarial flips, yielding a 12% test-accuracy improvement. Figure 2 details this over 28 initially poisoned nodes: 10 sanitized to ground truth, 7 relabeled to non-ground-truth classes, and 13 clean labels flipped.
• Dynamic versus one-shot correction. Because CoLAT updates the supervision signal each epoch, it adapts beyond the observed poison distribution; the paper contrasts this with PIGNN and CRGNN (which leave supervision unchanged), RTGNN, and CP (whose label restoration happens once before training).
• Scaling results on OGBN-ArXiv (169k nodes). With partitioned attacks, clean training falls to 0.4741 (NTK 30%) and 0.207 (SMulti 30%), while CoLAT reports 0.6184 and 0.2485. CoLAT operates over edges at O(|E|) cost, versus dense n×n adjacency reconstruction in PIGNN and O(n²) contrastive loss in CRGNN.
• Heterophilic graphs are harder. On Actor, CoLAT is roughly on par with clean training at low NTK poisoning (0.2756 vs 0.2849 at 5%) and better at 30% NTK (0.2381 vs 0.2179), with clearer SMulti gains: 0.2435 vs 0.1997 at 10% and 0.1444 vs 0.1079 at 30%.
• Architecture flexibility. On Cora ML with GAT and GraphSAGE backbones, CoLAT behaves similarly to the GCN case, with larger gains under stronger attacks; under SMulti, GraphSAGE exceeds 72% accuracy across all poisoning rates and gains up to 31% at 30% poisoning. CRGNN, also contrastive, does not adapt as well to architecture changes.
• The contrastive detector is the key component. Replacing it with a logit-based heuristic gives only a 0.4% gain on Cora ML at 30% NTK poisoning versus 13% with the detector, and about 5% versus up to 31% under SMulti.
• Iterative adversarial training beats one-shot filtering. Using the detector once as a filter (100 epochs on Cora ML, then early stopping) gives roughly 1% gain at 20% NTK poisoning versus about 13% for adversarial training; under SMulti at 20%, about 8% versus roughly 26%.
• Top-k selection direction matters. Selecting the most inconsistent nodes consistently outperforms selecting the most consistent nodes across attacks and poisoning rates on Cora ML (Table 4).
• Update interval is tunable for cost. On OGBN-ArXiv, updating the detector once every δ ∈ {1, 5, 10, 20} classifier steps still improves accuracy over clean training while reducing training time.

Methodology in Plain English

The defense is framed as a two-player game. The defender wants a classifier that performs well; the attacker wants to relabel a small number of training nodes (at most k) to hurt it as much as possible. Searching over every possible set of label flips is combinatorially infeasible, so the authors split the attacker's job into two easier questions: which nodes to flip, and what to flip them to.

For the first question they train a second GNN, the detector, with a contrastive objective defined on edges. Node embeddings are normalized to unit length, and for each edge the loss pulls same-label neighbors together while pushing different-label neighbors apart beyond a margin γ. A poisoned node's assigned label conflicts with its neighborhood, so it incurs a large contrastive penalty and ends up poorly aligned with its neighbors. Each labeled node is then scored by its average contrastive distance to its neighbors (normalized by degree to keep scores comparable across node degrees, and min–max normalized to [0, 1]), and the top-k scorers form the attack set. Crucially, the detector is trained separately from the classifier so that selection is not confounded by the classifier's current state.

For the second question, each selected node is given the label that maximizes the classifier's current cross-entropy loss, subject to being different from its current label — ensuring a genuine flip rather than a re-flip.

These steps alternate: update the classifier on the current adversarial labels, refit the detector, re-score nodes, reassign labels, repeat. Because the detector is refit on the adversarially modified labels, the procedure also implicitly corrects some originally poisoned labels, which the paper describes as sanitization. On large graphs, the detector can be updated less frequently than the classifier to save time.

Why This Matters

Impact on research. The paper targets a gap it identifies as under-explored: adversarial label poisoning in graphs, as opposed to the much-studied structural attacks. It also reframes contrastive learning on graphs — normally used to shape representations — as a mechanism for detecting corrupted nodes to drive adversarial training. The bilevel decomposition (node selection plus label assignment) is a template for making minimax defenses tractable where the perturbation space is combinatorial.

Real-world applications (as motivated by graph learning generally):

• Fraud detection on transaction and account-linkage graphs, where a handful of mislabeled accounts can distort a supervised detector.
• Social network abuse and bot/spam detection, where labels come from noisy or partially adversarial sources.
• Recommendation and user–item interaction graphs, where implicit feedback is unreliable.
• Citation and knowledge-graph curation, where annotation errors propagate through relational structure.
• Molecular property prediction, where wet-lab labels are expensive and sparse.

Industry relevance. Label noise is pervasive in production ML, and label poisoning is a concrete supply-chain and crowdsourcing threat. CoLAT's O(|E|) edge-based cost, its ability to run without architectural changes on large sparse graphs, and its demonstrated scaling to a 169k-node graph speak directly to deployment constraints, and its tunable detector update interval exposes an explicit training-time/accuracy trade-off.

Future Directions

• Heterophily. The paper's own framing: the contrastive score treats connected same-label nodes as positives, which is a sparse signal when adjacent nodes often differ in label. Adapting the objective to consider all same-label nodes would help heterophilic graphs but, the authors note, would largely remove the role of graph structure — an unresolved tension they probe in Appendix Table 15.
• Adaptive attacks. The evaluation uses attacks that approximate the bilevel problem with linear or kernel surrogates and do not significantly alter homophily. Robustness against an attacker aware of CoLAT's detection mechanism is not reported.
• Theoretical grounding. The minimax problem is approximated rather than solved; the paper offers a geometric intuition (poisoned nodes violate the contrastive structure) but no formal guarantee on detection or robustness.
• Beyond node classification and the tested settings. The framework is developed and evaluated for semi-supervised node classification with a flip budget on labeled nodes; extensions to other graph tasks and to joint structural-plus-label attacks are not reported.

Target Audience

Graduate students and researchers in graph machine learning and trustworthy ML who already know GNN message passing and adversarial training; practitioners building supervised models on relational data where training labels are untrusted or crowdsourced; and readers following the label-noise, data-poisoning, or contrastive-learning literatures who want a graph-specific treatment. Beginners will need background reading on GNNs and bilevel optimization before the formulation is accessible.

Authors’ abstract

Graph Neural Networks (GNNs) are effective for modeling relational data but are vulnerable to label poisoning, where a small number of corrupted training labels can propagate errors across the graph via message passing. Despite this risk, defenses against label poisoning remain underexplored: existing methods are primarily designed for label noise and often rely on robust losses or heuristic data cleaning that fail to distinguish adversarial poisoning from naturally hard examples. In this paper, we propose CoLAT (Contrastive Label-flipping for Adversarial Training), a novel contrastive adversarial training framework tailored for robust node classification on graphs. The core of our approach is a structure-aware detection model that uses contrastive learning to identify label-structure inconsistencies. Unlike prior contrastive methods that focus on representation learning, CoLAT leverages contrastive embeddings to select high-risk nodes that guide adversarial label-flipping during training. This alternating optimization not only performs structure-aware adversarial training but also implicitly sanitizes some corrupted labels, improving robustness against diverse attacks from the literature. Extensive experiments on multiple benchmarks show that CoLAT consistently outperforms existing defenses under various poisoning intensities while scaling efficiently to large graphs.