Beyond Suffixes: Token Position in GCG Adversarial Attacks on Large Language Models

AI summary

Overview

Research area: Adversarial robustness and safety alignment of Large Language Models, specifically gradient-based jailbreak attacks (GCG).

Technical level: Intermediate. Readers should be comfortable with tokenization, prompt-level attacks, attack success rate (ASR), and attention weights, though the paper explains its core idea in accessible terms.

Scope: The paper studies how the placement of GCG-optimized adversarial tokens (prefix vs. suffix, and evaluation position) changes jailbreak success rates in white-box and black-box cross-model settings.

What This Paper Is About

Almost all work on the Greedy Coordinate Gradient (GCG) jailbreak attack assumes the adversarial tokens belong at the end of the prompt, as a suffix. This paper asks whether that assumption is actually optimal, and treats the position of the adversarial token block inside the prompt as an attack axis in its own right. The goal is to show that both optimizing for a prefix and simply relocating an existing adversarial block change attack success rates, meaning that safety evaluations fixing tokens to one position can overstate robustness.

Key Contributions

1. The paper identifies adversarial token position as a previously underexplored attack axis in LLM jailbreak attacks, showing that it significantly affects jailbreak success. The authors define "adversarial token position" as the placement of the contiguous chunk of adversarial tokens within the prompt (e.g., as a prefix or suffix), not the position of an individual token.
2. It introduces GCG-prefix, a variation of GCG in which adversarial tokens are prepended to the prompt and optimized under the same objective as standard GCG (referred to as GCG-suffix), and evaluates both under a white-box setting.
3. It demonstrates that varying the position of optimized adversarial tokens within a prompt increases ASR in both white-box and black-box cross-model scenarios. In the cross-model setting, adversarial tokens generated on one model are tested on others to assess transferability.
4. It analyzes attention allocation across all transformer layers for prefix versus suffix attacks, finding that attention-based interpretations of ASR are position dependent and that the previously reported attention–ASR correlation is incomplete when restricted to middle-to-later layer analysis.

Main Findings

• No universally dominant position in white-box evaluation. Table 1 reports that when adversarial tokens are evaluated only at their optimized position, neither prefix- nor suffix-based optimization consistently dominates across the five models tested.
• Position variation raises ASR on Deepseek and Vicuna. Standard GCG-Suffix achieves higher ASR on deepseek-llm-7b-chat and vicuna-7b-v1.5, but GCG-Prefix overtakes once adversarial token position variation is allowed (ASR@k=2), increasing ASR from 10% to 15% for Deepseek and from 83% to 99% for Vicuna.
• Prefix optimization wins on Qwen. For Qwen2.5-7B-Instruct, GCG-Prefix consistently outperforms GCG-Suffix, with ASR@k=1 and ASR@k=2 increasing from 45% and 46% to 60% and 61%, respectively.
• Smaller but real gains on Mistral and Llama. For Mistral-7B-Instruct-v0.3, ASR@k=1 rises from 71% to 80% for GCG-Prefix and stays at 94% for GCG-Suffix; for Llama-2-7b-chat-hf, it rises from 35% to 36% (prefix) and from 40% to 41% (suffix). The authors note these are smaller increases but still indicate sensitivity to placement even when the optimization position is held fixed.
• Transferability also improves with position variation. In the cross-model transferability results (Table 2, ASR@k=2 with absolute increases from ASR@k=1 to ASR@k=2 in parentheses), neither GCG-Prefix nor GCG-Suffix consistently dominates; effectiveness varies with both source and target model. Allowing position variation increases transferability significantly, with several pairs showing large increases up to 49% (Deepseek, Mistral and Vicuna are called out).
• Zero-transfer cells exist. Several attack–target pairs report 0% ASR, including Llama and Deepseek attacks on Llama-2-7b-chat-hf as target and Qwen attacks on Llama as target.
• Attention shifts with position. For successful GCG attacks on Llama (Figure 1) and, per the appendix, on Vicuna, Qwen, Mistral and Deepseek, suffix-based attacks exhibit more attention on adversarial tokens when evaluated at later layers, consistent with prior work. When adversarial tokens are placed as a prefix, attention at most layers remains dominated by the goal (the original harmful prompt) tokens, despite successful jailbreaks.
• Layer range matters for attention analysis. The authors report that the apparent failure of attention as a saliency metric is partially, but not in all cases, mitigated when attention is analyzed across all transformer layers: later layers under-attend to adversarial prefixes while earlier layers allocate more attention to adversarial tokens.
• Qualitative failure-to-success cases. Appendix examples show an attack succeeding only when adversarial tokens are computed as a prefix and evaluated as a suffix (Figure 9), and another succeeding only when evaluated as a prefix (Figure 10).

Methodology in Plain English

The authors take the standard GCG attack as their baseline. GCG is a white-box attack that iteratively optimizes a discrete sequence of adversarial tokens by approximating token-level gradients and greedily updating individual tokens to minimize a loss defined on a specified target output sequence. Normally, that optimized block is appended to the end of the user prompt.

The researchers build two attack variants. GCG-suffix is the standard method: adversarial tokens appended to the end of the prompt. GCG-prefix prepends the initial adversarial tokens to the beginning of the prompt and optimizes them under the same objective. During evaluation, the same adversarial tokens are placed either at their optimized position or at the alternate position (prefix or suffix) while keeping the base prompt unchanged.

Both variants use identical hyperparameters: adversarial tokens are initialized as a string of twenty "x" tokens (length k = 20), optimization runs for 250 iterations with no early stopping, and at each step a single token is selected and replaced from a candidate set of size 256 based on gradient scores, with random seed 42. No additional separators or formatting tokens are introduced beyond standard whitespace.

Evaluation uses Attack Success Rate measured by an LLM judge (GPT-4), following the evaluation procedure suggested in Qi et al. (2024a); only samples with a harmfulness score equal to or above 5 are considered successful. ASR@k=1 means the adversarial prompt is evaluated at its optimization position; ASR@k=2 (Both) means it is successful when evaluated either as a prefix or as a suffix. All evaluations use greedy decoding (temperature 0) and a maximum generation length of 512 tokens, with each adversarial prompt evaluated once per configuration.

The test set is 100 harmful prompts randomly sampled from AdvBench. The models tested are deepseek-llm-7b-chat, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Llama-2-7b-chat-hf, and vicuna-7b-v1.5. For the transferability study, adversarial tokens generated on each model are tested on every target model.

Finally, the authors compute an attention score as the average of the attention weights of the adversarial or goal tokens on the GCG target output tokens, computed per decoder layer, extending a method used in Wang et al. (2025) beyond the suffix-only setting. They plot these scores for successful attacks across all layers.

Why This Matters

Impact on research. Much of the jailbreak literature implicitly assumes the suffix is the optimal adversarial token position, including work that prunes tokens or interprets attacks through attention dynamics. This paper argues that assumption creates a blind spot: evaluations that fix adversarial tokens to a single position overestimate adversarial robustness and underestimate real-world risk. It also cautions against relying on attention as a saliency metric for adversarial behavior, since prefix attacks succeed while later layers attend mostly to the goal tokens. The authors suggest attention should be augmented by views into internal model mechanisms such as refusal directions.

Real-world applications.

• Red-teaming and safety benchmarking: safety evaluations should vary the structural placement of adversarial tokens rather than testing one fixed prompt layout.
• Model deployment decisions: teams choosing between open-weight models should not assume that low ASR under a fixed suffix evaluation implies robustness.
• Guardrail and input-filtering design: filters keyed to detecting appended adversarial suffixes could miss adversarial content placed at the start of a prompt.
• Transferability threat modeling: since position variation increased transferability (up to 49% in some pairs), attacks optimized on one model may be more dangerous to other models than suffix-only evaluations suggest.

Industry relevance. The evaluated models are open-weight, widely deployed 7B chat and instruction-tuned systems from multiple vendors, and the experiments rely on a fixed user-message prompt template without special separators. This makes the positional effect easy to reproduce in standard deployment formats, and it directly affects how vendors and evaluators should design jailbreak benchmarks and robustness claims.

Future Directions

• Extend the analysis of adversarial token position to attacks beyond GCG and to attack taxonomies beyond the suffix-based framing.
• Systematically vary adversarial token position beyond just prefixes or suffixes (the authors state this as an explicit future aim).
• Investigate internal model mechanisms beyond attention, such as refusal directions, to better explain why position changes jailbreak success.
• Clarify why the attention–ASR relationship depends on which transformer layers are examined, since the paper reports that early layers allocate more attention to adversarial prefixes while later layers under-attend to them.
• Determine the conditions under which prefix optimization beats suffix optimization and vice versa, since the paper finds the winner varies with both source and target model and no position dominates consistently.

Target Audience

Safety and alignment researchers working on jailbreak attacks and adversarial robustness; red-teaming practitioners and benchmark designers who evaluate LLM refusal behavior; engineers deploying open-weight chat models who need realistic robustness estimates; and interpretability researchers studying attention-based explanations of adversarial behavior. The paper is also useful to graduate students entering the jailbreak literature, since it introduces GCG, its prefix variant, and the standard LLM-judge ASR protocol in a compact form.

Authors’ abstract

Large Language Models (LLMs) have seen widespread adoption across multiple domains, creating an urgent need for robust safety alignment mechanisms. However, robustness remains challenging due to jailbreak attacks that bypass alignment via adversarial prompts. In this work, we focus on the prevalent Greedy Coordinate Gradient (GCG) attack and identify a previously underexplored attack axis in jailbreak attacks typically framed as suffix-based: the placement of adversarial tokens within the prompt. Using GCG as a case study, we show that both optimizing attacks to generate prefixes instead of suffixes and varying adversarial token position during evaluation substantially influence attack success rates. Our findings highlight a critical blind spot in current safety evaluations and underline the need to account for the position of adversarial tokens in the adversarial robustness evaluation of LLMs.