Universal and Transferable Attacks on Pathology Foundation Models

AI summary

Overview

Research area: Adversarial machine learning and computational pathology, specifically the security and robustness of pathology foundation models.

Technical level: Intermediate. Readers need some familiarity with machine learning, foundation models, and medical image analysis.

Scope: This paper introduces a single, fixed adversarial noise pattern that can degrade the performance of multiple pathology foundation models and their downstream clinical tasks, even when those models were never seen during attack development.

What This Paper Is About

Pathology foundation models are large pretrained AI systems used for many medical image tasks, such as classifying tissue, detecting disease, or supporting diagnosis. This paper asks whether these models share a common vulnerability that can be exploited with one carefully designed but visually subtle perturbation. The goal is to show that such an attack can be both universal across different image fields of view and transferable to unseen black-box pathology models, revealing a broad security risk.

Key Contributions

1. Introduces Universal and Transferable Adversarial Perturbations (UTAP), a fixed and weak noise pattern optimized with deep learning to disrupt pathology foundation model feature representations.
2. Demonstrates universality: the same perturbation works across diverse field-of-views and does not depend on the dataset used to develop the attack.
3. Demonstrates transferability: the perturbation degrades external, black-box pathology foundation models that were never seen during optimization.
4. Systematically evaluates UTAP across multiple state-of-the-art pathology foundation models and datasets, showing significant downstream performance drops with visually imperceptible image modifications.

Main Findings

• Universal perturbation: A single fixed noise pattern disrupts feature representations across different fields of view and datasets.
• Transferable black-box attack: The same perturbation successfully degrades pathology foundation models that the attackers never had access to.
• Downstream performance loss: Models affected by UTAP show performance drops in downstream tasks, including misclassification across a wide range of unseen data distributions.
• Visually imperceptible changes: The adversarial modifications are subtle to human observers but effective against the models.
• Broad threat: UTAP is not limited to one model or one dataset, suggesting a general vulnerability across emerging pathology foundation models and their applications.
• Robustness benchmark: The attack establishes a high-standard evaluation target for model robustness and can provide assets for adversarial training and defense development.

Methodology in Plain English

The researchers used deep learning to optimize a single small noise pattern that, when added to pathology images, confuses pathology foundation models. They then tested whether this same pattern could affect many different image crops, datasets, and other pathology models that were not used during optimization. By comparing downstream task performance with and without the noise, they measured how much the perturbation degraded model accuracy and reliability. The noise was kept visually subtle so that the changes would not be obvious to a human pathologist.

Why This Matters

Impact on research: The paper shows that pathology foundation models can share transferable vulnerabilities, which challenges assumptions that large pretrained medical AI systems are inherently robust. It sets a benchmark for evaluating and improving medical AI security.

Real-world applications:

• Clinical pathology decision support tools that use foundation models to classify tissue or detect disease.
• Cancer detection, tumor subtyping, and grading systems built on digital pathology images.
• Whole-slide image analysis platforms used in hospitals and research labs.
• Medical AI deployment pipelines where a single compromised or corrupted input pattern could affect many downstream tasks.

Industry relevance: Medical AI vendors, hospital systems, regulators, and foundation model developers need to account for adversarial threats before deploying pathology AI in clinical settings. Robustness testing and adversarial training may become part of regulatory and quality-control workflows.

Future Directions

• Develop effective defenses, such as adversarial training, robust feature encoders, input purification, and attack detection methods.
• Investigate why adversarial perturbations transfer so effectively across different pathology foundation models and training datasets.
• Evaluate UTAP in realistic clinical workflows, including whole-slide images, multi-modal pathology data, and larger patient populations.
• Create standardized robustness benchmarks and certification procedures for pathology foundation models before clinical deployment.

Target Audience

This paper benefits machine learning researchers, computational pathology scientists, medical AI developers, and regulators who are concerned with the safety, robustness, and reliability of foundation models in pathology. It is also useful for clinical pathologists and hospital informatics teams evaluating AI tools for diagnostic use.

Authors’ abstract

We introduce Universal and Transferable Adversarial Perturbations (UTAP) for pathology foundation models that reveal critical vulnerabilities in their capabilities. Optimized using deep learning, UTAP comprises a fixed and weak noise pattern that, when added to a pathology image, systematically disrupts the feature representation capabilities of multiple pathology foundation models. Therefore, UTAP induces performance drops in downstream tasks that utilize foundation models, including misclassification across a wide range of unseen data distributions. In addition to compromising the model performance, we demonstrate two key features of UTAP: (1) universality: its perturbation can be applied across diverse field-of-views independent of the dataset that UTAP was developed on, and (2) transferability: its perturbation can successfully degrade the performance of various external, black-box pathology foundation models - never seen before. These two features indicate that UTAP is not a dedicated attack associated with a specific foundation model or image dataset, but rather constitutes a broad threat to various emerging pathology foundation models and their applications. We systematically evaluated UTAP across various state-of-the-art pathology foundation models on multiple datasets, causing a significant drop in their performance with visually imperceptible modifications to the input images using a fixed noise pattern. The development of these potent attacks establishes a critical, high-standard benchmark for model robustness evaluation, highlighting a need for advancing defense mechanisms and potentially providing the necessary assets for adversarial training to ensure the safe and reliable deployment of AI in pathology.