Angular Gradient Sign Method: Uncovering Vulnerabilities in Hyperbolic Networks

AI summary

Overview

• Research area: Adversarial machine learning and non-Euclidean (hyperbolic) representation learning.
• Technical level: Intermediate — assumes familiarity with adversarial attacks (FGSM/PGD) and basic concepts of hyperbolic geometry, but the core idea is intuitive.
• Scope: Proposes and evaluates a geometry-aware adversarial attack that perturbs only the angular (semantic) component of gradients in hyperbolic embedding spaces, across image classification and cross-modal retrieval.

What This Paper Is About

Standard adversarial attacks like FGSM and PGD were designed for models whose internal representations live in flat Euclidean space. Recent "hyperbolic networks" instead embed data (images, text, hierarchies) in curved hyperbolic space, where the geometry is fundamentally different. This paper shows that attacking these models in the usual way is wasteful, and introduces a method that exploits hyperbolic structure directly to produce stronger, more semantically meaningful adversarial examples.

Key Contributions

1. Diagnosis of geometric mismatch: The authors argue that conventional attacks are suboptimal for hyperbolic networks because they ignore curvature and the hierarchy-preserving structure of hyperbolic embeddings.
2. AGSM (Angular Gradient Sign Method): A new single-step attack that computes the gradient in the tangent space, decomposes the resulting representation shift into a radial (hierarchical depth) component and an angular (semantic) component, and perturbs the input using only the angular direction.
3. PAGD (Projected Angular Gradient Descent): A multi-step iterative extension of AGSM modeled on PGD, which repeatedly maximizes angular displacement while staying inside a perturbation budget.
4. Broad empirical validation: Demonstrations on Poincaré ResNet (CIFAR-10/100, Tiny ImageNet) and HyCoCLIP (MS COCO, Flickr30K for text-to-image and image-to-text retrieval), showing consistent gains over FGSM and PGD, plus analysis of embedding distances, confidence drops, ℓ₂-norm variants, and adversarial training.

Main Findings

• Radial perturbations are nearly harmless: On Poincaré ResNet-32 with CIFAR-100, a purely radial shift left top-1 accuracy at 53.44%, identical to the clean baseline. A purely angular shift alone dropped it to 25.56%, showing that almost all the damage from FGSM (19.67%) comes from angular movement.
• AGSM beats FGSM everywhere: Across ResNet-20 and ResNet-32 on all three image datasets, AGSM consistently cut robust accuracy by roughly 5–13 percentage points more than FGSM at the same perturbation budget. Example: CIFAR-100 at ε = 8.0/255 on ResNet-32, FGSM left 19.67% accuracy while AGSM left 13.93%.
• PAGD outperforms PGD: The multi-step angular attack dropped accuracy by roughly 1–10 percentage points more than standard PGD, with the largest gains on CIFAR-10.
• Retrieval tasks are also more vulnerable: On HyCoCLIP with ViT-S/16 at ε = 3.2/255 on COCO, AGSM reduced text-to-image Recall@5 to 8.20% versus FGSM's 11.20% (clean: 55.10%). With ViT-B/16, AGSM's extra degradation reached about 5% in image-to-text Recall@5.
• Perturbed embeddings travel farther: Measured under the Lorentz hyperbolic distance, AGSM-perturbed features ended up consistently farther from the originals than FGSM-perturbed ones (e.g., 0.4457 vs. 0.3883 on COCO at ε = 8.0/255).
• Confidence collapses harder: AGSM caused larger drops in maximum softmax probability than FGSM, and the gap widened as ε increased (e.g., 0.5597 vs. 0.4364 on CIFAR-10 at ε = 8.0/255).
• Not tied to one norm: Under an ℓ₂ constraint, AGSM still outperformed FGSM on both CIFAR-10 and CIFAR-100, indicating the angular vulnerability is not an artifact of the ℓ∞ constraint.
• Adversarial training is only partially effective: Training on AGSM-perturbed examples improved robustness to angular attacks but not uniformly, and it cost more clean accuracy than FGSM-based augmentation. On CIFAR-100, AGSM augmentation raised robustness against AGSM to 25.66% but lowered clean accuracy to 46.96% from 49.63%.

Methodology in Plain English

The authors start from a simple observation about hyperbolic space: distance from the center of the ball encodes how "general" or "specific" a concept is (the hierarchy level), while moving around the circle at a fixed radius encodes fine-grained semantic variation within that level. These are the radial and angular directions.

Their attack works as follows. First, they run a normal FGSM step to see how the model's internal representation moves. They then split that movement into the part that goes straight out from the center (radial) and the part that goes sideways (angular). Since radial changes barely affect predictions, they discard that part and keep only the sideways direction. They then backpropagate this angular direction through the network to the input image, take the sign of the resulting gradient, and apply a small perturbation in that direction. The result is an image that mainly shifts the model's representation sideways — across semantic categories — rather than pushing it up or down the hierarchy.

For the iterative version (PAGD), they repeat this process over multiple small steps, each time recomputing the angular direction, and project the image back into the allowed perturbation ball after each step, mirroring how PGD extends FGSM. The method works for both the Poincaré ball and the Lorentz hyperboloid models, using standard logarithmic maps to move between the manifold and its tangent space.

Why This Matters

• Impact on research: This is one of the first works to treat adversarial robustness as a geometric problem in curved representation spaces. It establishes that the radial/angular decomposition is a meaningful axis of vulnerability analysis, giving the robustness community a new lens for evaluating non-Euclidean models and a baseline that future defenses must beat.
• Real-world applications:
  • Safety-critical image classifiers that use hierarchy-aware embeddings (e.g., fine-grained species or defect classification).
  • Vision-language retrieval systems used in search or content moderation, where an attacker could subtly manipulate captions to return semantically wrong results.
  • Recommendation and knowledge-graph systems that rely on hyperbolic embeddings of taxonomies, where adversarial manipulation could shift items across categories undetected.
  • Any deployment of hyperbolic models in which robustness claims currently rest only on Euclidean attack evaluations.
• Industry relevance: As hyperbolic architectures move from research prototypes toward production (particularly in multimodal retrieval and hierarchical recommendation), teams need attack tools that actually stress-test them. AGSM is a cheap, drop-in white-box evaluation that reveals vulnerabilities that standard FGSM/PGD testing would miss, which directly affects how much confidence practitioners can place in existing robustness benchmarks.

Future Directions

• Geometry-aware defenses: The paper's adversarial training experiments show only modest robustness gains with clean-accuracy cost, so the open question is what a defense tailored specifically to angular perturbations would look like.
• Black-box and transferability: Only white-box attacks are evaluated; whether angular perturbations transfer across models or survive under query-limited access remains untested.
• Extending beyond L∞ and L₂: It is unclear whether other constraints (e.g., perceptual or semantic budgets) yield the same angular advantage, or whether an adaptive angular attack could be even stronger.
• Broader architectures and modalities: Applying the radial/angular decomposition to hyperbolic graph networks, knowledge-graph embeddings, or fully hyperbolic vision transformers would test how general the finding is.

Target Audience

• Adversarial robustness researchers working on attacks or defenses beyond Euclidean settings.
• Practitioners deploying hyperbolic models for hierarchical classification, cross-modal retrieval, or recommendation who need realistic robustness evaluations.
• Graduate students and engineers with a working knowledge of FGSM/PGD and an interest in non-Euclidean representation learning.
• Theoreticians studying how curvature affects gradient behavior, since the radial/angular split offers a clean formal handle on that question.

Authors’ abstract

Adversarial examples in neural networks have been extensively studied in Euclidean geometry, but recent advances in \textit{hyperbolic networks} call for a reevaluation of attack strategies in non-Euclidean geometries. Existing methods such as FGSM and PGD apply perturbations without regard to the underlying hyperbolic structure, potentially leading to inefficient or geometrically inconsistent attacks. In this work, we propose a novel adversarial attack that explicitly leverages the geometric properties of hyperbolic space. Specifically, we compute the gradient of the loss function in the tangent space of hyperbolic space, decompose it into a radial (depth) component and an angular (semantic) component, and apply perturbation derived solely from the angular direction. Our method generates adversarial examples by focusing perturbations in semantically sensitive directions encoded in angular movement within the hyperbolic geometry. Empirical results on image classification, cross-modal retrieval tasks and network architectures demonstrate that our attack achieves higher fooling rates than conventional adversarial attacks, while producing high-impact perturbations with deeper insights into vulnerabilities of hyperbolic embeddings. This work highlights the importance of geometry-aware adversarial strategies in curved representation spaces and provides a principled framework for attacking hierarchical embeddings.