Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Models

AI summary

Overview

Research area: LLM security and efficient inference — specifically, how depth-based dynamic inference (early-exit / layer-skipping models) changes a model's vulnerability to optimization-based jailbreak attacks.

Technical level: Intermediate. It assumes familiarity with transformer layer stacks, early-exit architectures, and token-level adversarial suffix optimization (GCG), but the core argument is explained conceptually.

Scope: A single-attack, single-dynamic-inference-method empirical study comparing jailbreak robustness, attack optimization behavior, and downstream utility across three intermediate exit points (layers 15, 18, 21) and the full 32-layer model for two LLaMA backbones.

What This Paper Is About

Large language models can be made cheaper to run by letting them produce answers from an intermediate layer instead of the final one, and this "depth-based dynamic inference" lets one model serve several compute budgets. Separately, LLMs are known to be breakable by adversarial suffixes that bypass safety alignment. The paper asks the question that sits between these two facts: if a model is allowed to answer from a shallower depth, does that make it harder or easier to jailbreak, and does the answer depend on which model you started from?

Key Contributions

1. First systematic study of jailbreak robustness under depth-based dynamic inference. The authors state that prior optimization-based jailbreak work has only been studied against standard full-depth inference, and they extend the evaluation to intermediate exits.

2. A controlled experimental design using Balcony. They deliberately choose the Balcony method because its backbone is frozen and only lightweight exit modules are trained, so any change in robustness across exits can be attributed to the dynamic-inference mechanism rather than to the base model's safety alignment being altered.

3. A three-axis analysis rather than a single ASR number. They measure attack success rate, GCG optimization loss over 150 steps, and utility on six multiple-choice benchmarks under both clean and adversarially suffixed inputs, then relate these axes to each other.

4. A backbone-dependence finding with a concrete counterexample. They show that shallow-exit robustness is not a property of the exit mechanism but of the backbone, contrasting Balcony-LLaMA3-8B (large robustness gain at shallow exits) with Balcony-LLaMA2-7B (little gain), despite identical exit configuration, training corpus, and self-distillation objective.

The code is released at a public GitHub repository (LassNajmuldeen/balcony-jailbreaking).

Main Findings

• Shallow exits are more robust than deep ones, but the size of the effect is backbone-dependent. Across both models and both benchmarks, attack success rate is highest at the full model. For Balcony-LLaMA3-8B, ASR on HarmBench drops from 28% at the full model to 4% at exit-15, and on AdvBench from 28% to 2%. For Balcony-LLaMA2-7B, the effect is far smaller: HarmBench ASR runs from 36% at exit-15 to 44% at the full model, and AdvBench from 46% to 54%.

• The two backbones show qualitatively different vulnerability profiles. Balcony-LLaMA2-7B's ASR increases gently and near-monotonically with depth (HarmBench: 36%, 40%, 40%, 44% for exit-15, exit-18, exit-21, full; AdvBench: 46%, 42%, 48%, 54%), suggesting vulnerability is distributed across depth. Balcony-LLaMA3-8B climbs slowly (HarmBench 4%, 12%, 12%, then 28%; AdvBench 2%, 0%, 12%, then 28%), with a sharp jump only at the full model — suggesting its GCG vulnerability is concentrated in later layers.

• Deeper exits are easier to optimize against. GCG loss decreases monotonically with exit depth in all four model–benchmark settings, with the ordering full < exit-21 < exit-18 < exit-15 holding throughout the 150 optimization steps. For Balcony-LLaMA3-8B the final loss at the full model is reported as 3× lower than at exit-15.

• The depth separation in optimization is much sharper for LLaMA3. On Balcony-LLaMA2-7B, the four loss curves converge into a narrow band after roughly 50 iterations, with exit-15 only modestly behind. On Balcony-LLaMA3-8B, exit-15 stays clearly separated for all 150 iterations and converges to a substantially higher loss, while exit-18, exit-21, and the full model cluster together.

• Clean utility rises with depth. On all six benchmarks the authors report clean utility increasing monotonically with exit depth. For Balcony-LLaMA3-8B, HellaSwag goes from 0.519 at exit-15 to 0.792 at the full model, ARC-C from 0.334 to 0.540, and MMLU from 0.310 to 0.621; for Balcony-LLaMA2-7B, HellaSwag goes from 0.630 to 0.760 and MMLU from 0.369 to 0.409.

• Adversarial suffixes barely dent utility at any depth. Comparing the clean and attacked tables, scores are largely stable across exits and benchmarks for both models. The stated exception is ARC-E on the 7B model, where the suffix appears to interfere with the task itself (for example, exit-15 drops from 0.617 to 0.482 and the full model from 0.739 to 0.572) rather than merely eliciting unsafe behavior.

• Utility cannot be used as a proxy for robustness. An exit can be simultaneously low-capability and highly jailbreakable (LLaMA2-7B at exit-15), and conversely a low ASR at a shallow exit cannot be dismissed as a broken sub-model, since clean utility at those exits remains substantial. The authors conclude that the apparent safety of shallow exits on LLaMA3 is tied to how little task-relevant and instruction-following capability has formed at that depth, rather than to a distinct, depth-localized safety mechanism.

Methodology in Plain English

The researchers took two pretrained models with 32 transformer layers each — LLaMA2-7B (6.5B non-embedding parameters, 2T pretraining tokens) and LLaMA3-8B (6.9B, 15T) — and turned each into a dynamic model using Balcony. In Balcony the original weights stay frozen; only small "exit" modules are trained (via KL self-distillation on Cosmopedia V2, 31.5B tokens, batch size 256, 30K steps, sequence length 4,096, peak learning rate 5×10⁻⁴, cosine schedule). The same three exit locations, {15, 18, 21}, were used for both models, so that differences between the two could not be blamed on the training recipe.

They then ran Greedy Coordinate Gradient (GCG), a token-level attack that searches for a suffix to append to a harmful request so that the model is more likely to start its reply with an affirmative phrase. The attack was run for 150 steps on 50 harmful behaviors per benchmark, on HarmBench and AdvBench. Attack success rate is the fraction of behaviors that the HarmBench judge scores above 0.5, and they also recorded the mean final cross-entropy loss the attack reached at each exit.

To check whether lower attack success at shallow exits was just a sign of a weaker model, they separately measured general capability on six multiple-choice benchmarks — HellaSwag, ARC-C, ARC-E, WinoGrande, PIQA, and MMLU — using lm-eval-harness, reporting acc_norm for HellaSwag, ARC, and PIQA and acc for WinoGrande and MMLU. Each was measured both on clean questions and with each exit's strongest GCG suffix appended to every question. Finally, they crossed the three axes — depth, attack loss, utility, attack success — to see whether they move together and whether that coupling is consistent across the two backbones.

Why This Matters

Impact on research. Dynamic inference is usually evaluated on efficiency-versus-accuracy trade-offs, with safety treated as inherited from the full model. This paper shows that the safety profile is not inherited uniformly: the same exit configuration yields a dramatic robustness difference on one backbone and almost none on another. It also argues methodologically that training-side confounds must be controlled (hence the frozen-backbone choice) before attributing a robustness change to the inference mechanism.

Real-world applications.

• Edge and on-device assistants that must run under tight memory or latency budgets and therefore answer from shallow exits.
• Cloud serving with tiered latency or cost tiers, where the same model is exposed at several compute levels to different customers.
• Industrial and safety-sensitive deployments where a model's refusal behavior is part of the compliance case, and where the deployed configuration may not be the full-depth one.
• Safety evaluation pipelines and red-teaming suites for any model that ships with selectable inference depth, since a single full-model evaluation would miss the exit-specific behavior.

Industry relevance. Providers increasingly sell one model at multiple price and latency points. The paper's results imply that each selectable depth is effectively a separately exposed endpoint with its own attack surface, and that the cheapest configuration is not automatically the safest or the most dangerous — it depends on the backbone's own pretraining. It also implies that attack loss at a given exit can be used as a rough early signal of how exploitable that exit is.

Future Directions

• Test adaptive and stronger attacks. The authors note their findings come from a single attack with fixed exit points; attacks that explicitly optimize against a chosen exit, or attacks beyond GCG (including prompt-level attacks), could change the picture.
• Study routing policies rather than fixed exits. The paper evaluates a fixed set of exit points; input-adaptive routing would decide depth per prompt, which could change both the attack surface and the utility trade-off.
• Analyze internal representations. The authors explicitly leave representational analysis for future work, which is the natural next step for testing their hypothesis that LLaMA2's exploit-relevant features are already present early in the network while LLaMA3's form mainly in later layers.
• Extend to more backbones and exit configurations. Since the headline effect was large on one model family and small on another, the finding needs replication across more model families, more exit depths, and different training corpora before it can be used as a design rule.

Target Audience

Researchers and engineers working on efficient LLM inference who need to know what safety properties they inherit when they deploy an early-exit model; LLM security researchers who want a template for evaluating attacks beyond full-depth inference; and practitioners building safety or red-teaming evaluations for models offered at multiple compute tiers. It is most useful to readers with some background in transformer architectures and adversarial prompt optimization, though the empirical comparisons and the utility tables are accessible to a broader technical audience.

Authors’ abstract

Large Language Models (LLMs) have been shown to be computationally expensive and vulnerable to jailbreak attacks, in which adversarial prompts bypass safety alignment and induce harmful responses. To address this inefficiency, a range of techniques have been proposed to enable efficient LLM inference. Depth-based dynamic inference has demonstrated promising results by enabling outputs to be generated from intermediate layers. It offers the flexibility to adjust model behavior based on the available resource budget. However, how this flexibility affects the robustness of LLMs against jailbreak attacks remains unexplored. In this paper, we address this gap via a comprehensive investigation into the impact of depth-based dynamic inference on the robustness of LLMs against jailbreak attacks. Specifically, we make the following key observations: 1) Shallow exits are consistently more robust to jailbreak attacks than deeper ones; 2) Exit depth affects how easily the adversarial suffix can be optimized, with deeper exits reaching lower attack loss; and 3) Adversarial suffixes affect utility uniformly across exit depths. These insights can help design dynamic inference methodologies that are more robust to jailbreak attacks. Code is available at https://github.com/LassNajmuldeen/balcony-jailbreaking