Hidden State Poisoning Attacks against Mamba-based Language Models

AI summary

Overview

Research area: Adversarial robustness and security of large language models, specifically state space models (SSMs) such as Mamba and Mamba-2 and hybrid SSM–Transformer architectures.

Technical level: Intermediate. The core idea is intuitive (a short phrase that makes the model forget), but the paper includes a formal derivation of the hidden-state contraction bound, a mechanistic interpretability study of internal activation norms, and genetic-algorithm optimization of adversarial triggers.

Scope: The paper introduces and quantifies Hidden State Poisoning Attacks (HiSPAs) — short trigger sequences that overwrite the recurrent hidden state of Mamba-based language models and destroy long-context information retrieval — and introduces RoBench-25 as a benchmark for measuring this effect across models.

What This Paper Is About

Mamba and Mamba-2 are state space models that replace attention with a recurrence, giving them linear time complexity and an explicit hidden state that acts as memory. The paper asks whether that design creates a security weakness: can a short, innocuous-looking phrase inserted into a prompt permanently overwrite what the model has already read? The authors show that it can, calling the phenomenon a Hidden State Poisoning Attack, and they build a benchmark (RoBench-25) and an interpretability analysis to measure and explain it.

Key Contributions

1. A new attack class. The authors define Hidden State Poisoning Attacks (HiSPAs), adversarial triggers that exploit SSM recurrence to irreversibly overwrite hidden states in Mamba-based models (including Mamba-2), with a zero-shot black-box variant (Z-HiSPA) and an optimized white-box variant (M-HiSPA).

2. RoBench-25. A robustness benchmark for long-context information retrieval built from 120 NeurIPS 2025 paper titles plus abstracts (260.8 tokens on average), each paired with 2 True/False questions, scored with the clipped Heidke skill score (CHSS), which ranges from 0 (random guess) to 1 (perfect).

3. Architecture-specific vulnerability demonstrated at scale. Z-HiSPAs cause severe degradation in Mamba but not in Transformers, and optimized M-HiSPAs can collapse the 52B-parameter Jamba-1.7-Mini hybrid, whose architecture is 7/8 Mamba blocks.

4. A mechanistic explanation and a link to prompt injection. The authors identify a narrow band of Mamba blocks (28–37) whose output-embedding norms correlate almost perfectly with failure (block 29: r = −0.9707), and show that HiSPA-like prefixes amplify Prompt Injection Attack success on the updated Open-Prompt-Injections benchmark for SSM-based models. Code and data are released at https://github.com/TortueSagace/hispa.

Main Findings

• Mamba collapses under HiSPAs. Under the configuration with neither awareness nor recovery instructions, HiSPA triggers produce a 62% average CHSS drop from clean to poisoned prompts for Mamba-2.8b.

• Transformers do not. The exact same configuration gives Pythia-2.8b a 41% average CHSS increase, which the authors hypothesize relates to distractors mitigating Transformers' miscalibration in question-answering.

• The degradation is statistically real. Mamba answered 0.615 of 960 clean questions correctly, above the 0.5 random baseline (binomial test, p = 6.3 × 10⁻¹³). Using a paired Wilcoxon signed-rank test over 120 draws with Holm correction across the seven triggers, five of seven triggers produced significant degradation (all p < 10⁻⁴), with accuracy drops of 5.0 to 8.1 points. Triggers A and G moved Mamba by fewer than 2 accuracy points and were not significant after correction (p = 0.083).

• A ceiling-level Transformer is equally unaffected. SmolLM3-3B (instruction fine-tuned) had a clean CHSS of 1.000, a per-trigger average of 0.998 (minimum 0.996), and no individual cell below 0.983 across 120 draws, 4 configurations and 7 triggers — so the asymmetry is not a floor effect from Pythia's low baseline.

• Forgetting semantics are not required. In a larger trigger corpus (189 additional triggers), a benign fake completion ("…flagged as safe and permanently kept in my state.") cost Mamba 5.4 accuracy points, close to the 7.5 points lost to its explicitly forgetting twin ("…flagged as unsafe and permanently deleted…"). The fake-completion structure carries most of the effect.

• Jamba resists zero-shot attacks but not optimized ones. Jamba-1.7-Mini shows near-perfect clean CHSS and no Z-HiSPA degradation. However, a GA-discovered 12-token trigger containing the <|endoftext|> special token (loss −0.9998) collapses Mamba-2.8B to random guessing, and also collapses Pythia-2.8B and Jamba-1.7-Mini to zero CHSS, while Llama-3.3-70B-Instruct, Llama-3.1-8B-Instruct and SmolLM3-3B resisted. The authors note this mixed trigger likely exploits special-token effects on attention as well as SSM recurrence.

• Block-1 saturation is necessary but not sufficient. Most GA-generated M-HiSPA triggers reached lower first-block loss than Z-HiSPAs (below −0.998) yet produced smaller RoBench-25 degradation. Optimizing instead for the block-29 output L2 norm produced a nonsensical but effective trigger reaching CHSS 0.117, outperforming 6 of 7 Z-HiSPA triggers.

• A mid-layer amplification band. Blocks 28–37 show strong negative linear correlations with CHSS (r < −0.91 for all blocks in the band), with blocks 28, 29, 30 and 35 strongest; block 29 has the highest absolute correlation (r = −0.9707). This supports a two-stage failure mechanism: hidden-state saturation at the first Mamba block, then mid-layer amplification.

• HiSPAs amplify prompt injection on hybrids. On the updated Open-Prompt-Injections benchmark (restricted to the first 3 datasets: sentiment analysis, spam detection, duplicate sentence reduction), Jamba's attack success value rose from 0.490 (naive) to 0.743 ("combine"), i.e. +0.253 absolute and ×1.516 relative. Llama-3.1-8B-Instruct rose more mildly (+0.107 absolute, ×1.265), while Llama-3.3-70B-Instruct became less vulnerable (−0.107 absolute, ×0.607), a reversal the authors attribute to the "ignore" component of the combined prompt rather than the HiSPA-like prefix.

• Mamba-2 behaves differently but remains vulnerable. Its scalar-identity state transitions make it more robust to the zero-shot triggers A–G (its movements there are mixed, small, and near an already low clean baseline), but a targeted search in the appendix finds fake-completion triggers that drive it to CHSS 0.

Methodology in Plain English

The attack exploits how Mamba stores memory. The model keeps a hidden state that is updated token by token, and a learned gate controls how much of each new token enters that memory. If a run of tokens drives the gate into a strongly "contracting" regime, the old state decays geometrically while the new input is written in at full strength, so the earlier content is effectively erased. The authors derive the conditions under which a short trigger of a handful of tokens can reliably dominate the stored state, and show the same argument carries over to Mamba-2, where the state transition is scalar-identity rather than a general diagonal.

To test this, they place prompts in the shape: target abstracts (InT), then an optional trigger, then distractive abstracts (DiT), then questions about the targets. Prompts can also include an awareness instruction up front (warn the model to ignore forgetting instructions) and/or a recovery instruction before the questions. They score answers with CHSS, and check that Mamba still emits valid True/False answers so that degradation reflects retrieval failure rather than format breakdown.

For trigger construction, the zero-shot setting (Z-HiSPA) uses natural-language strategies borrowed from Liu et al. (2024): escape, ignore, and fake completion. The white-box setting (M-HiSPA) uses a genetic algorithm with tournament selection, single-point crossover and per-token mutation (population 10, tournament size 3, elite size 4, crossover rate 0.7, mutation rate 0.15, up to 10,000 generations, i.e. 100,000 queries per seed), optimizing either the first-block objective or the block-29 output norm. An early verification used a greedy heuristic over 6-token triggers, brute-forcing 1,000 random tokens from a 50,000-token vocabulary per position, across 100 random seeds (42–141) on 10 Tesla V100-SXM3-32GB GPUs. Model comparisons use Mamba-2.8b and Pythia-2.8b, matched in size, context window (2,048 tokens) and training data (The Pile), plus Jamba-1.7-Mini and SmolLM3-3B; small models ran on a Tesla V100-SXM3-32GB and Jamba on an NVIDIA H200 NVL.

Why This Matters

Impact on research. The paper argues that adversarial robustness is an under-examined dimension of SSM research, that no dedicated benchmark existed before RoBench-25, and that SSM or hybrid models should not be evaluated on perplexity and downstream accuracy alone. It connects a previously separate line of work on prompt injection to the internal recurrence of SSMs, and it produces an interpretability signal (the mid-layer block band) that already informed a follow-up defense, Clasp, reported as reaching 99.3% document-level and 95.9% token-level F1, retaining 91.6% against structurally novel triggers, at 1,032 tokens per second under 4GB of VRAM.

Real-world applications:

• Retrieval over scientific literature. The authors cite assisted literature review tools such as Scispace and Elicit, and note that malicious actors have already embedded prompt injections directly inside papers (Keuper, 2025), creating a realistic attack surface for SSM-based pipelines.
• Agentic deployments. Contextual forgetting of previously stated constraints can cause guardrail degradation, authorization errors, confidentiality leaks from lost redaction tags, and integrity failures in planning from attacker-skewed state.
• Document screening pipelines. The Clasp defense is evaluated on résumé screening, a domain where hidden-state corruption could hide or alter applicant information.
• Adversarial deployment environments generally. Any long-context SSM or hybrid deployment exposed to untrusted input — the paper's recommendation is robustness testing before deployment.

Industry relevance. Hybrid SSM–Transformer designs are moving into production, with the Jamba family reaching up to 398B parameters and Mamba blocks making up 7/8 of Jamba's architecture. The finding that a handful of tokens can collapse a 52B model's benchmark score, and can amplify prompt injection success by ×1.516, is directly relevant to anyone deploying these architectures in adversarial settings. The energy argument also matters: SSMs are attractive because Transformers' quadratic complexity is an ecological concern, so the efficiency gain is only worthwhile if the deployed system cannot be collapsed by a few tokens.

Future Directions

• Turning the interpretability signal into a deployed defense. The paper treats the near-perfect block-29 correlation as a concrete target for lightweight activation-based monitoring that requires no adversarial training, and develops a detector (Clasp) separately at the document and token level.

• Causal isolation of the mid-layer band. The authors note that their evidence is correlational plus optimization-based, and that isolating the block-29 activation causally would require holding the input fixed and manipulating that activation alone, for example by rescaling its norm while preserving its direction.

• Broadening the evaluation beyond binary retrieval. Impact on open-ended generation, multi-hop reasoning, multilingual input, conversational memory and agentic workflows is not measured, because separating the hidden-state effect from confounds such as instruction-following and decoding behaviour needs a benchmark designed for that purpose, as does extending the study to model families beyond Mamba and Mamba-2.

• A symmetric fine-tuned SSM baseline. The authors state that no instruction fine-tuned open-source Mamba of comparable size exists to their knowledge, so their claim that HiSPAs are architectural rests on the mechanism rather than on that model comparison. The reported attacks are also described as a lower bound, since longer triggers or objectives defined on deeper representations could only be stronger.

Target Audience

This paper is most useful to: (1) security and red-teaming researchers working on adversarial robustness of LLMs; (2) machine learning engineers and architects evaluating or deploying Mamba, Mamba-2, Jamba, Nemotron or other SSM-based hybrids in products exposed to untrusted input; (3) interpretability researchers interested in how Mamba layers consolidate factual content; and (4) prompt-injection researchers who want to understand how injection behavior differs between attention-based and recurrent architectures. Readers unfamiliar with state space models will need the background in Section 2, but the central argument is accessible without deep mathematical training.

Authors’ abstract

State space models (SSMs) like Mamba offer efficient alternatives to Transformer-based language models, with linear time complexity. Yet, their adversarial robustness remains critically unexplored. This paper studies the phenomenon whereby specific short input phrases induce a partial amnesia effect in such models, by irreversibly overwriting information in their hidden states, referred to as a Hidden State Poisoning Attack (HiSPA). Our benchmark RoBench-25 allows evaluating a model's information retrieval capabilities when subject to HiSPAs, and confirms the vulnerability of SSMs against such attacks. Even the recent Jamba-1.7-Mini SSM--Transformer (a 52B hybrid model) collapses on RoBench-25 under some HiSPA triggers, whereas pure Transformers do not. We also observe that HiSPA triggers significantly weaken the Jamba model on the popular Open-Prompt-Injections benchmark, unlike pure Transformers. We further show that the theoretical and empirical findings extend to Mamba-2, and also analyse a Mamba-2-based hybrid (Nemotron-3-Nano). Finally, our interpretability study reveals patterns in Mamba's hidden layers during HiSPAs that could be used to build a HiSPA mitigation system. The full code and data to reproduce the experiments can be found at https://github.com/TortueSagace/hispa.