Semantic Router: On the Feasibility of Hijacking MLLMs via a Single Adversarial Perturbation

AI summary

Overview

Research area: AI security and adversarial machine learning, specifically attacks on Multimodal Large Language Models (MLLMs), with connections to computer vision and embodied/agentic systems.

Technical level: Advanced. The paper assumes familiarity with adversarial perturbations, latent-space embeddings, Jacobians, Lipschitz constants, and gradient-based optimization.

One-sentence scope: The paper tests whether a single, input-agnostic adversarial perturbation can act as a "semantic router" — one perturbation that reads the semantics of whatever image it is applied to and routes the model to a different attacker-chosen target for each semantic category.

What This Paper Is About

MLLMs deployed in autonomous driving and robotics often make stateless decisions: at each timestep the model sees only the current frame plus a prompt and emits an immediate action, with no memory of previous steps. While each decision is independent, their accumulation determines the agent's physical trajectory. This paper asks whether one universal perturbation can hijack many of these stateless decisions at once, forcing the model to output a different predefined target depending on the visual semantics of the current frame. The authors name this threat Semantic-Aware Hijacking and the perturbation a Semantic-Aware Universal Perturbation (SAUP).

Key Contributions

1. Demonstrating feasibility of many-to-many hijacking. The paper shows a single perturbation can hijack stateless decisions and analyzes the perturbation in the latent space both theoretically and empirically, decomposing it into a "Dominant Shift" and a "Semantic Deflection."
2. The SORT algorithm. A Semantic-Oriented optimization strategy that combines Normalized Space Optimization (NSO) and Semantic Separation Optimization (SSO).
3. The RIST dataset. Real Image Sequence Trajectories, a newly annotated dataset with fine-grained semantic labels drawn from continuous videos for autonomous driving and robotic scenarios.
4. Extensive evaluation. Experiments across three MLLMs and two datasets of differing semantic granularity, achieving a 66% attack success rate over five targets using a single frame against Qwen.

Main Findings

• Single perturbation, multiple targets on Qwen. One perturbation is sufficient to mislead Qwen, achieving success rates of 93%, 77%, 61%, and 66% under 2, 3, 4, and 5 targets, respectively; the headline result is 66% attack success rate over five targets using a single frame against Qwen.

• Fine-grained semantics are attackable but harder. On the RIST dataset, SAUPs achieve an average ASR of 72% on the RoboTasking test set (2 targets) and 62% on the AutoDriving test set (5 targets).

• Overfitting on a small training set. On the Intern model, ASR reaches 100% on the training set but drops to 61% on the test set for the AutoDriving scenario. The paper attributes this gap to the limited size of the RIST training set (50 images). Error bars in the RIST results are relatively large due to environmental diversity (lighting, backgrounds) and non-uniform semantic granularity across trajectories.

• Precise control over long outputs. In the 2-target setting with 10 words per target, SAUPs achieve an average ASR of 83% on the test set. With 5 targets, SAUPs still control 4 words per target at 54% average test ASR. At 15 words in the 2-target setting, ASR reaches 78% on Qwen and 83% on Intern.

• Information capacity limits the attack. On Qwen under the frame constraint, ASR drops from 93% (2 targets) to 31% (9 targets). The frame perturbation occupies 7,056 pixels and maintains a relatively high ASR (e.g., 42% on Llava for 9 targets), while the corner perturbation contains only 1,600 pixels and its ASR collapses to 0% on Llava and Qwen when the number of targets reaches 8.

• NSO is the critical component. Removing NSO causes ASR to plummet by an average of 49.5% on the test set; specifically on Qwen with 5 targets and the frame constraint, ASR drops from 66% to near zero. The loss curve for the "w/o NSO" variant remains high and fails to descend, indicating a poor local minimum.

• SSO helps most in crowded multi-target settings. The "w/o SSO" variant performs comparably to the default at 2 targets, but SSO yields an average ASR gain of 6% as semantic crowding increases; without it, optimization converges early but gets trapped in a local optimum.

• Latent-space geometry is confirmed empirically. Using Llava-1.5-7B with images from five ImageNet classes and PCA visualization, a "pure perturbation" (all-zero image plus δ) lands at the centroid of the perturbed clusters, confirming the Dominant Shift; perturbed features form distinct separable sub-clusters by semantics, confirming Semantic Deflection; and a confidence heatmap shows higher confidence along the diagonal, confirming alignment with proxy target embeddings.

• Theoretical bound. Theorem 3.4 states that if the maximum required expansion ratio exceeds the global Lipschitz constant of the vision encoder, no perturbation exists that can map the input set to the target set. As inputs become more similar, the required expansion ratio grows toward infinity, making the bound harder to satisfy.

Methodology in Plain English

The attacker collects images belonging to several distinct semantic classes and assigns each class a specific target sentence. A single perturbation is then trained on this collection.

To make the mapping succeed, the authors use two ideas. First, Normalized Space Optimization (NSO): instead of optimizing the perturbation directly in pixel space [0, 1], they optimize a variable in a normalized space and invert the normalization to recover the pixel-space perturbation. This preconditioning keeps step sizes consistent and avoids the loss plateauing. Second, Semantic Separation Optimization (SSO): they combine a standard cross-entropy loss (which pushes toward the correct target) with a margin-based loss that forces the confidence gap between the correct target and competing targets to exceed a margin m. The total objective is cross-entropy plus λ times the margin loss, where λ balances target alignment against semantic separation.

The perturbations are constrained to be small: a frame of width 6 pixels, or corner patches of size 20 × 20 pixels. A fixed prompt, "Describe this image", is used with greedy decoding across all models. Evaluation uses Attack Success Rate (ASR), which requires output tokens to match the target sequence in both content and order. The analysis is conducted under a digital white-box setting. For the RIST dataset, Gemini-2.5-pro was used to assign contextually relevant actions as targets based on scenario safety constraints.

Why This Matters

This work extends adversarial attacks from single-input or single-target settings to a many-to-many universal mapping, a capability the paper describes as a "switch-case" attack. Because stateless MLLM decisions accumulate into physical trajectories, hijacking a chain of them can produce cascading, dangerous outcomes rather than isolated errors. The paper's comparison table distinguishes SAUPs from traditional adversarial perturbations (per-instance, untargeted or single-targeted) and from universal adversarial perturbations (which are either untargeted or many-to-one) by their generalization and semantic awareness.

Real-world applications and risk surfaces:

• Autonomous driving. A perturbation embedded in a frame could steer a vehicle toward attacker-chosen commands such as "accelerate," "merge," or "steer" at precise moments, cumulatively guiding it toward a predefined destination.
• Robotic manipulation. In the RIST RoboTasking scenario, an attacker could program a robot to "grasp" when it sees one object and "throw" when a human enters the scene.
• MLLM-based computer and smartphone agents. A malicious frame in a web advertisement could coerce an agent to first upload a user's private data and then execute a "submit" command to finalize the breach.
• Physical deployments generally. The paper frames the threat as relevant to any stateless MLLM perception-and-action loop, including OpenVLA-style architectures that create a new context at each timestep.

Industry relevance: The findings are directly relevant to developers of autonomous driving stacks, robotics platforms, and agentic interfaces that rely on open-source MLLMs. The paper argues that even under white-box assumptions, a widely deployed open-source model can be targeted with a crafted SAUP, making the vulnerability a practical rather than purely theoretical concern.

Future Directions

• Physical-world robustness. The authors state an intent to bridge digital simulation and reality by developing SAUPs resilient to physical conditions such as brightness changes, and note that robustness to lighting and motion blur requires further study. They also explicitly acknowledge that physical deployment remains a significant challenge.
• Relaxing the white-box assumption. The paper relies on full access to the model's architecture and parameters and identifies black-box settings as an open limitation.
• Understanding and improving the local linearity approximation. The theoretical analysis depends on a local linear assumption that introduces non-negligible error as input magnitude grows; the authors quantify relative error and feature similarity across three MLLMs in an appendix but describe the framework as providing qualitative insight rather than precise quantitative predictions.
• Scaling capacity and data. Because overfitting is attributed to the limited RIST training set (50 images), larger and more diverse training data is a natural next step, alongside studying how increasing the perturbed area (explored in an appendix) affects the information bottleneck.

Target Audience

This paper is best suited for adversarial machine learning and AI security researchers, MLLM and vision-language model developers, and safety engineers working on autonomous driving, robotics, or agentic systems. Reviewers and practitioners interested in threat modeling for embodied AI will also find the latent-space analysis and the RIST dataset useful. Readers need a background in adversarial optimization and representation geometry to follow the theoretical sections; the experimental results and threat framing are more broadly accessible.

Note: the paper content provided is truncated partway through Appendix A, so the full design details, size, and composition of the RIST dataset are not fully reported in the available text.

Authors’ abstract

Multimodal Large Language Models (MLLMs) are increasingly deployed in stateless systems, such as autonomous driving and robotics. This paper investigates a novel threat: Semantic-Aware Hijacking. We explore the feasibility of hijacking multiple stateless decisions simultaneously using a single universal perturbation. We introduce the Semantic-Aware Universal Perturbation (SAUP), which acts as a semantic router, "actively" perceiving input semantics and routing them to distinct, attacker-defined targets. To achieve this, we conduct theoretical and empirical analysis on the geometric properties in the latent space. Guided by these insights, we propose the Semantic-Oriented (SORT) optimization strategy and annotate a new dataset with fine-grained semantics to evaluate performance. Extensive experiments on three representative MLLMs demonstrate the fundamental feasibility of this attack, achieving a 66% attack success rate over five targets using a single frame against Qwen.