One Token Embedding Is Enough to Deadlock Your Large Reasoning Model

AI summary

Overview

Research area: Adversarial machine learning and AI security, specifically resource-exhaustion attacks against large reasoning models (LRMs).

Technical level: Intermediate to advanced. The paper assumes familiarity with token embeddings, chain-of-thought reasoning, gradient-based optimization, and backdoor/poisoning threat models.

Scope (one sentence): The paper proposes and evaluates the "Deadlock Attack," a two-phase method that learns a single adversarial token embedding and implants it as a backdoor trigger so that a poisoned reasoning model generates endlessly and exhausts its inference-time compute budget.

What This Paper Is About

Large reasoning models solve problems by generating long chains of thought, and this iterative thinking is what makes them good at math and coding — but it also means a malicious input could keep them thinking forever. The authors ask whether a minimal perturbation, specifically a single token embedding, can hijack a model's reasoning loop and force it into a perpetual "deadlock" state that burns through any available token budget. Their goal is to build a practical, stealthy attack that survives the jump from continuous embedding space to real discrete text, and to show it works across multiple models and benchmarks.

Key Contributions

1. Initiating the study of resource-exhaustion attacks on LRMs: The paper identifies test-time computational scaling and the reasoning dynamics of chain-of-thought models as a previously underexplored adversarial attack surface.
2. The Deadlock Attack (DA): An efficient method that uses a single adversarial token embedding to hijack the model's reasoning pathway, inducing perpetual thinking loops that consume the full generation budget.
3. Identifying and diagnosing the continuous-to-discrete projection gap: The authors show that naïvely projecting an optimized adversarial embedding to the nearest vocabulary tokens neutralizes the attack, and they analyze this failure using linear mode connectivity (LMC), Gaussian-smoothing-based robust optimization, and iterative projection.
4. A backdoor carrier mechanism: They embed the optimized adversarial vector directly into the model's embedding matrix as the representation of a predefined trigger token, enabling reliable activation through ordinary text while leaving benign behavior intact.
5. Extensive evaluation: Four LRMs (Phi-RM, Nemotron-Nano, R1-Qwen, R1-Llama) across GSM8K, MATH500, MMLU-Pro (Math), and additional stealthiness and robustness evaluations.

Main Findings

• Perfect attack success rate: DA achieves 100% attack success rate (ASR) across all four evaluated LRMs and all three reasoning benchmarks (GSM8K, MATH500, MMLU-Pro Math), where success is defined as generation reaching the 4000-token limit. Baseline models occasionally show non-zero ASR on their own (for example, 14.0% for Phi-RM on MMLU-Pro Math and 6.98% for Phi-RM and R1-Qwen on MATH500).
• Token counts saturate at the cap: Every DA-activated model consistently generated 4000 tokens on every benchmark, versus baseline averages such as 525 tokens (R1-Qwen, GSM8K) and 1631 tokens (Phi-RM, MMLU-Pro Math).
• Inference time inflates substantially: For example, Phi-RM rose from 25.31 s (GSM8K baseline) to 118.41 s under DA, and R1-Qwen rose from 13.77 s to 106.01 s.
• Initial proof of concept: On the first 50 samples of GSM8K with R1-Llama, the unattacked model had 2.0% ASR, 921 average tokens and 26.45 s average time, while the DA variant reached 100.0% ASR, 4000 average tokens and 102.7 s.
• The 4000-token cap is an experimental constraint, not an attack limit: The authors state the attack is designed to escalate until any imposed budget is consumed, and they report an extended evaluation with the token limit raised to 20,000 on the AIME 2024 benchmark (Appendix C) where the attack maintains a near-perfect success rate.
• Robust to overthinking mitigations: Against Chain of Draft (CoD), Concise Chain-of-Thought (CCoT), and NoThinking on GSM8K, ASR remained 100.0% for all four models under all three strategies. The paper reports the accompanying column values as, for example, 118.41 (no mitigation), 117.31 (CoD), 105.08 (CCoT), and 103.44 (NoThinking) for Phi-RM.
• Stealthy on benign inputs: With the trigger absent, accuracy changed only marginally relative to the baseline models on the evaluated test sets (50 samples each for GSM8K and MMLU-Pro, and the 43-sample Level 1 subset of MATH500). Notable cases include Phi-RM changing from 94.0 to 96.0 on GSM8K, R1-Qwen from 80.0 to 82.0 on GSM8K and from 90.7 to 93.0 on MATH500 (L1), while some entries decreased (for example, R1-Llama from 93.0 to 83.7 on MATH500 L1). A larger stealthiness evaluation using 500 samples per benchmark across six benchmarks is reported in Appendix C.
• Projection failure is well characterized: In the LMC analysis (L = 1, N = 20 training samples, evaluated on 10 test samples), attack loss stays low at and near the interpolation endpoints when embeddings are interpolated directly, but rises sharply and stays high across the entire interpolation range once each interpolated embedding is projected to its nearest token embeddings.
• Gaussian smoothing and iterative projection both fail: The embedding still converged with Gaussian noise standard deviations up to 0.2 (reference: the pretrained embedding layer's parameter standard deviation is approximately 0.02), but post-projection loss stayed high at all noise levels. Projecting every K = 300 steps produced recurring, non-diminishing loss spikes. Alternative metrics (L1-norm, cosine similarity) and PCA before projection also failed (Appendix B).
• Trigger design: The default trigger is the single token "!!!!!" with adversarial embedding length L = 1, chosen because it is rare and semantically neutral; homoglyph-based alternatives are discussed in Appendix A.
• The attack can be activated with a qualitative loop: In the demonstrative example in Section 3.4, the model's reasoning falls into a deadlock loop and is forced to generate until reaching the maximum token limit of 20000.

Methodology in Plain English

The attack has two phases.

Phase 1 — Learn a malicious embedding. The attacker assumes white-box access to the victim model's parameters. They prepend a short trainable vector sequence (the adversarial embedding) to the input and optimize it with gradient descent. The training objective is simple: at every punctuation mark that normally ends a reasoning step (like "." or "?"), maximize the probability that the model's next token comes from a set of hesitant or reflective words such as "Wait" or "But." In other words, train the model, through its input, to keep second-guessing itself instead of concluding. Training used 20 (problem, answer) pairs derived from the first 30 level-5 MATH500 samples (with 100 reasoning answers generated per sample by R1-Qwen), a 10-sample validation set, the Adam optimizer, learning rate 10^-3, and 1000 steps. The objective is input-agnostic and does not target specific questions.

Phase 2 — Turn the embedding into a usable trigger. Because a real attacker can only send text, the learned continuous embedding must become discrete tokens. The authors show that simply snapping each vector to its nearest vocabulary embedding destroys the attack: the projection error is larger than the perturbation the embedding can tolerate. Instead of fighting this gap, they sidestep it: they define a trigger token sequence and overwrite the model's embedding matrix entries for those tokens with the optimized adversarial vectors. The poisoned model is then released publicly; when a victim deploys it, sending the trigger in any query activates the endless-reasoning behavior, while normal queries behave as before.

Evaluation. They measure Attack Success Rate (share of inputs hitting the 4000-token cap), average tokens generated, and average inference time. Stealthiness is measured by accuracy on benchmarks with the trigger absent. Robustness is measured by re-running the attack under CoD, CCoT, and NoThinking decoding and prompting strategies.

Why This Matters

Impact on research. The paper reframes overthinking — normally treated as an efficiency problem to be optimized away — as a security vulnerability that can be weaponized. It also documents a concrete failure mode of continuous-relaxation adversarial attacks on LLMs (the continuous-to-discrete projection gap) and offers the backdoor carrier as a workaround, which is relevant to the broader discrete-adversarial-attack literature.

Real-world applications and risks (as described in the paper):

• AI supply chain poisoning: An attacker can fine-tune, poison, and release a seemingly enhanced open-source reasoning model on public hubs, which downstream developers or cloud providers then deploy.
• Denial-of-service against LLM services: A remotely sent trigger query can trap a deployed model in endless generation, disrupting service for other users and consuming compute.
• Economic and latency cost attacks: Because providers may support long-form generation with high token limits (the paper cites 32k as an example), resource consumption can escalate far beyond the 4000-token experimental cap.
• Evading standard quality gates: Because benign-input behavior is nearly unchanged and the trigger is unknown to the victim, standard evaluation protocols are unlikely to flag the poisoned model.

Industry relevance. Any organization that hosts or distributes reasoning models — model hubs, cloud inference providers, and enterprises deploying open-weight models — faces this supply chain and resource-exhaustion risk. The paper's suggestion is that defenses may need to go deeper than prompting or decoding tricks, for example explicitly identifying and unlearning implanted adversarial trigger embeddings.

Future Directions

• Trigger-embedding detection and removal: The authors explicitly point to unlearning or identifying implanted trigger embeddings as a promising defense direction, since prompt-level and decoding-level mitigations failed.
• Scaling the threat to production token limits: The paper validates the attack at a 20,000-token cap on AIME 2024; behavior under much larger production budgets (the paper mentions 32k) is not evaluated in the provided content.
• Generalizing beyond single-token triggers and math reasoning: The default setting uses L = 1 and "!!!!!"; the paper discusses homoglyph triggers and reports an ablation on embedding length and training-set size in Figure 5, but the full results and the rest of the ablation section are truncated in the provided content, so the specific outcomes are not reported here.
• Broader mitigation strategies: The authors note existing overthinking mitigation methods fail against DA, leaving open the question of what defenses can neutralize reasoning-level hijacking without degrading legitimate long-form reasoning.

Target Audience

Machine learning security researchers studying adversarial attacks, backdoors, and LLM robustness; LLM infrastructure and deployment engineers responsible for serving or vetting open-weight models; model-hub and supply-chain risk teams; and efficiency researchers working on chain-of-thought overthinking, who will find their optimization problem reframed as an attack surface. Readers without a background in embeddings and adversarial optimization will need to consult the related-work section for context.

Authors’ abstract

Modern large reasoning models (LRMs) exhibit impressive multi-step problem-solving via chain-of-thought (CoT) reasoning. However, this iterative thinking mechanism introduces a new vulnerability surface. We present the Deadlock Attack, a resource exhaustion method that hijacks an LRM's generative control flow by training a malicious adversarial embedding to induce perpetual reasoning loops. Specifically, the optimized embedding encourages transitional tokens (e.g., "Wait", "But") after reasoning steps, preventing the model from concluding its answer. A key challenge we identify is the continuous-to-discrete projection gap: naïve projections of adversarial embeddings to token sequences nullify the attack. To overcome this, we introduce a backdoor implantation strategy, enabling reliable activation through specific trigger tokens. Our method achieves a 100% attack success rate across four advanced LRMs (Phi-RM, Nemotron-Nano, R1-Qwen, R1-Llama) and three math reasoning benchmarks, forcing models to generate up to their maximum token limits. The attack is also stealthy (in terms of causing negligible utility loss on benign user inputs) and remains robust against existing strategies trying to mitigate the overthinking issue. Our findings expose a critical and underexplored security vulnerability in LRMs from the perspective of reasoning (in)efficiency.