Certified but Fooled! Breaking Certified Defences with Ghost Certificates

AI summary

Overview

Research area: Adversarial machine learning, specifically attacks against certified robustness defenses (randomized smoothing).

Technical level: Advanced. The paper assumes familiarity with randomized smoothing, adversarial examples, and certified robustness. Concepts are explained here, but the underlying mathematics is graduate-level.

One-sentence scope: The paper introduces GhostCert, a region-based adversarial attack that tricks certified classifiers into both misclassifying an image and issuing a large (but misleading) robustness certificate, using perturbations far smaller and more imperceptible than prior work.

What This Paper Is About

Certified defenses like randomized smoothing promise provable guarantees: if a classifier issues a certificate with a large radius for an input, then no small perturbation should be able to change the model's prediction. A prior attack called Shadow Attack showed this promise could be broken, but only by applying large, visible perturbations. This paper asks whether the same spoofing can be achieved with much smaller, essentially invisible changes—and answers yes, using region-focused perturbations that concentrate on the semantically meaningful parts of an image.

Key Contributions

1. A new certificate-spoofing attack (GhostCert). The authors introduce a region-based approach that builds "salient-region masks" by combining object segmentation boundaries (from SAM) with gradient- or attention-based saliency maps, then constrains perturbations to those regions. This preserves image semantics and keeps distortions imperceptible.

2. Targeted certificate spoofing. Going beyond the untargeted attacks of prior work, the authors demonstrate attacks that force a certified model to issue a certificate for a chosen target class—a harder problem—and quantify the additional denial-of-service (abstention) behavior these attacks induce.

3. Large-scale evaluation against state-of-the-art certified defenses. GhostCert is tested on ImageNet against three defenses: a single ResNet-50 under randomized smoothing, an ensemble of three consistency-trained ResNet-50s, and DensePure (a diffusion-based denoiser combined with a BEiT transformer).

4. Human perceptual study. A crowd-sourced study (Amazon Mechanical Turk) confirms that GhostCert's adversarial images are rated as more natural-looking than those from the Shadow Attack across distortion budgets.

Main Findings

• Higher attack success at lower distortion. GhostCert consistently achieves higher Attack Success Rates than the Shadow Attack and its bounded variant, across all noise levels and perturbation budgets, while using smaller L2 perturbation norms.

• Ensemble defenses are substantially weakened. Against a consistency-trained ensemble under randomized smoothing at σ = 0.25, GhostCert reaches roughly 100% ASR (untargeted) and over 80% ASR (targeted), while Shadow variants plateau near 40% and 20% respectively.

• Spoofed certificates exceed source-image radii. GhostCert's forged certification radii frequently match or exceed the genuine certified radius of the original image—meaning the attack produces "strongly certified" misclassified inputs. The authors call these ghost certificates.

• DensePure remains the strongest defense but is still broken. Under DensePure, GhostCert achieves 30–100% success versus 30–65% for Shadow attacks, though its spoofed radii are slightly lower or comparable. The authors conclude DensePure is the most effective of the three defenses tested.

• Perturbations can induce abstention. In targeted settings where spoofing is difficult, GhostCert often pushes the certifier to abstain (a denial-of-service outcome), and its abstention rate generally exceeds that of Shadow variants.

• Imperceptibility is significantly improved. A user study found 58–74% of participants (varying by distortion budget) judged GhostCert's images more natural than Shadow Attack's.

• The certificates are not invalidated. The authors stress that their attack does not break the mathematical correctness of the certificate—it exploits the assumption that a large certified radius implies a trustworthy label. They frame the work as a cautionary tale rather than a full break of certification theory.

Methodology in Plain English

The attack proceeds in two stages:

1. Find the important regions. Using Segment Anything (SAM), the method produces candidate object regions (segmentation masks). A saliency map from GradCAM (for CNNs) or attention maps (for transformers) identifies which pixels the model weighs most heavily for its decision. Each region is scored by how much its pixels overlap with the saliency map, and the top k regions (k = 5 by default) are combined into a single "salient-region mask."

2. Perturb only inside those regions. A projected-gradient-descent (PGD) loop is run, but at every step the perturbation is multiplied by the mask, so only pixels inside the chosen regions can change. Crucially, the loss is computed over many noisy copies of the image (Monte Carlo samples, N = 1000) rather than a single version—this mimics how randomized smoothing actually classifies, and it is what allows the attack to manipulate the certifier, not just the underlying classifier.

The optimizer is adjusted depending on the defense: for a single model it targets the base classifier; for an ensemble it targets the averaged logits; for a denoised model it targets the composition of the denoiser and classifier. In targeted mode, the gradient is flipped and the loss is computed against a chosen target label.

The key design insight is that concentrating perturbations inside natural object boundaries is both more effective and less visible than constraining the perturbation globally—because the model's decision really does depend on those regions.

Why This Matters

Impact on research. The paper shows that the threat model for certified defenses must expand beyond "can an attacker change the label?" to "can an attacker also change the certificate?" It establishes that small, imperceptible, region-focused perturbations—not just large global ones—suffice to spoof robustness guarantees, and it introduces targeted certificate spoofing and certification-induced denial-of-service as distinct threat surfaces.

Real-world applications at risk:

• Autonomous driving systems that rely on certified perception models for safety-critical decisions.
• Medical imaging classifiers where a certificate might be read as clinical assurance.
• Content moderation and malware detection, where adversaries could obtain false assurances that a malicious input is robustly classified as benign.
• Regulatory compliance, where certification may be cited as evidence of robustness in high-stakes deployments.

Industry relevance. Any deployment that treats "certified accuracy" or "certified radius" as an indicator of trustworthiness—rather than just a mathematical property of the smoothed classifier at that point—should treat this paper as a warning. The authors explicitly recommend interpreting certificates as an indicator of label correctness only with caution.

Future Directions

• Certification schemes robust to spoofing. Can certificate issuance be made resistant to region-based, certifier-aware perturbations—for example by detecting when an input has been pushed near a decision boundary in a way that produces an anomalously large radius?
• Closing the DensePure gap. DensePure proved the strongest of the defenses tested. Understanding why diffusion-based denoising resists the attack better than ensembles could point toward stronger certified methods.
• Defense against abstention attacks. DoS via forced abstention was observed but not defended against; methods to distinguish adversarial abstention from legitimate uncertainty remain open.
• Extension to other modalities and threat models. The attack is demonstrated on ImageNet vision classifiers; whether region-focus spoofing generalizes to audio, text, or 3D inputs under different smoothing distributions is untested.

Target Audience

This paper is most valuable to adversarial machine learning and certified robustness researchers, security engineers evaluating the trust assumptions of certified defenses, and practitioners in safety-critical domains (autonomy, healthcare, security) who may be considering certification as part of a deployment decision. Readers without a background in randomized smoothing will benefit from the conceptual framing but should consult the cited foundational work (Cohen et al. 2019) for the underlying theory.

Authors’ abstract

Certified defenses promise provable robustness guarantees. We study the malicious exploitation of probabilistic certification frameworks to better understand the limits of guarantee provisions. Now, the objective is to not only mislead a classifier, but also manipulate the certification process to generate a robustness guarantee for an adversarial input certificate spoofing. A recent study in ICLR demonstrated that crafting large perturbations can shift inputs far into regions capable of generating a certificate for an incorrect class. Our study investigates if perturbations needed to cause a misclassification and yet coax a certified model into issuing a deceptive, large robustness radius for a target class can still be made small and imperceptible. We explore the idea of region-focused adversarial examples to craft imperceptible perturbations, spoof certificates and achieve certification radii larger than the source class ghost certificates. Extensive evaluations with the ImageNet demonstrate the ability to effectively bypass state-of-the-art certified defenses such as Densepure. Our work underscores the need to better understand the limits of robustness certification methods.