Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models

AI summary

Overview

Research area: Natural Language Processing / mechanistic interpretability of large language models, specifically the internal causes of chain-of-thought (CoT) reasoning. Technical level: Advanced (assumes familiarity with transformer hidden states, activation steering, and sparse autoencoders). Scope: The paper identifies sparse latent features in LLM activations that are causally tied to reasoning, and shows that steering those features at the first generation step induces CoT-like reasoning without CoT prompting.

What This Paper Is About

Chain-of-thought prompting reliably improves LLM reasoning, but it is unclear whether explicit step-by-step prompting is the only way to activate reasoning, or merely one trigger for an internal reasoning state. The authors use Sparse Autoencoders (SAEs) to find a small number of latent features whose activations differ between direct and CoT prompting, then test causally whether steering those features can produce reasoning behavior on its own. The goal is to show that reasoning is a latent internal mechanism that can be triggered through targeted intervention on internal representations rather than through prompt wording alone.

Key Contributions

1. A two-stage framework using SAEs to (a) discover candidate reasoning-related latent features through differential analysis of direct versus CoT prompting, and (b) causally validate them via targeted latent steering on a held-out test set.
2. Empirical evidence across six models from three families (LLaMA-3, Qwen-3, Gemma-3, up to 70B parameters) and three benchmarks (GSM8K, GPQA Diamond, BBH logical_deduction_three_objects) that steering one or a few identified features under direct prompting produces accuracy comparable to CoT while generating shorter outputs.
3. A direct comparison against dense representation steering (mean CoT-minus-direct activation difference), showing SAE-based sparse steering substantially outperforms dense steering, and in some cases dense steering fails entirely while sparse steering gives large gains.
4. Controlled experiments establishing causality and specificity: suppression of the top-25 identified features under CoT prompting impairs accuracy; the features respond to diverse reasoning-encouraging prompts but not to pseudo-CoT formatting; and identified features generalize across datasets.

Main Findings

• Steering induces reasoning without CoT prompts. On GSM8K, LLaMA-3.1-8B-Instruct rises from 24.5 accuracy (direct) to 73.3 (steered direct) and 79.3 (CoT); LLaMA-3.3-70B-Instruct rises from 46.7 to 88.8 (steered direct) versus 96.1 (CoT). On BBH, LLaMA-3.1-8B-Instruct goes from 50.8 (direct) to 61.6 (steered direct) versus 77.2 (CoT).
• Steered direct can match or exceed CoT in some settings. Gemma-3-4B-Instruct reaches 65.6 on BBH under steered direct versus 50.0 under CoT; Gemma-3-12B-Instruct reaches 95.6 on BBH under steered direct versus 95.2 under CoT; Qwen3-0.6B reaches 60.6 on GSM8K under steered direct versus 59.7 under CoT.
• Single-feature, first-step intervention suffices for large models. On LLaMA-3.3-70B-Instruct, steering a single latent feature (feature #13709) at only the first generation step achieves CoT-comparable or better accuracy with substantially fewer generated reasoning tokens.
• Steering acts as a trigger, not a refinement. Steered CoT yields only marginal improvements over standard CoT (for example, LLaMA-3.1-8B-Instruct GSM8K: 84.1 steered CoT versus 79.3 CoT; Qwen3-4B GSM8K: 62.5 versus 61.1), consistent with the feature already being strongly activated under CoT.
• SAE steering beats dense steering. On Gemma-3-4B-Instruct GSM8K, direct is 8.4, dense steering 16.2, SAE steering 74.0. On Qwen3-4B GSM8K, direct is 34.5, dense 34.6, SAE 60.0. Dense steering also transfers poorly to BBH (LLaMA-3.1-8B-Instruct: dense 56 versus SAE 61.6).
• Suppression under CoT impairs reasoning causally. Using Qwen3-4B and the top-25 features at layer 29, GSM8K CoT accuracy drops from 61.1 to 12.1 and BBH from 98.8 to 62.8, while output length does not collapse (GSM8K tokens 421 to 479; BBH 458 to 401). GPQA moves slightly up, from 18.8 to 21.7.
• Early, transient activation dynamics. The identified features peak early in decoding and decay rapidly; CoT prompting consistently induces stronger activation than direct decoding. Examples shown are feature #8629 in LLaMA-3.1-8B-Instruct and feature #13709 in LLaMA-3.3-70B-Instruct.
• Not tied to wording or verbosity. Alternative reasoning-encouraging prompts ("Explain how", "Solve carefully", "Think") activate the same feature, sometimes more strongly than step-by-step prompting, while a pseudo-CoT prompt that mimics step-by-step formatting without reasoning activates it less than direct prompting.
• Limited gains where multi-step reasoning matters little. On GPQA, where CoT does not consistently outperform direct decoding, steering gives little or no improvement (LLaMA-3.1-8B-Instruct: direct 28.7, steered direct 28.2, CoT 20.7). The GPQA-identified feature improves GSM8K and BBH but not GPQA itself.
• Partial cross-dataset overlap. Top-25 feature sets show Jaccard overlap of 0.11 (GSM8K–GPQA), 0.09 (GSM8K–BBH), and 0.28 (GPQA–BBH), indicating partially shared but substantially dataset-dependent feature discovery.
• Steering overrides reasoning-suppressing prompts. On Qwen models whose prompts include the no_think control token designed to suppress CoT, steered direct decoding still produces coherent multi-step reasoning.

Methodology in Plain English

The authors treat the model's hidden activations as the object of study rather than its text output. For a given question, they run the model twice: once with a plain direct prompt and once with a CoT prompt. They record activations at a chosen mid-to-late layer and pass them through a pretrained Sparse Autoencoder, which turns a dense hidden vector into a sparse set of latent feature activations. For each latent feature, they compute its average activation under CoT prompting minus its average activation under direct prompting, producing a differential score. The features with the largest absolute difference are the candidates.

They focus on the first generation step, based on the hypothesis and observation that reasoning-relevant activations appear early and later tokens dilute the signal. Candidate features are then tested one at a time: the model's latent activation for a single feature is increased additively by a fixed multiple of that feature's average magnitude, converted back through the SAE decoder, and injected into the model as a residual correction (the difference between the steered reconstruction and the plain reconstruction) to avoid reconstruction bias. Features whose singleton intervention consistently improves behavior on held-out training data are selected and frozen. Steering layer, feature index, and strength are tuned per model on a validation split and reused across all datasets. Evaluation uses GSM8K test, GPQA Diamond, and the BBH logical deduction task. For the causal-suppression test, the team projects the positive activations of the top-25 features back into hidden space and subtracts a scaled version of that contribution from the residual stream throughout prompt processing and generation.

SAE sources: Goodfire pretrained SAEs for LLaMA models, Gemma Scope 2 for Gemma models, and SAEs trained from scratch for Qwen models. All models are inference-only with frozen weights, and features are identified using 1,000 question–answer pairs from the GSM8K training split.

Why This Matters

This work reframes CoT prompting as one trigger among several for an internal reasoning state, and it provides intervention-level causal evidence rather than purely correlational probe results. That shifts interpretability research from describing representations to controlling them, and it suggests a route to reasoning behavior that does not require long verbal chains—steered direct outputs are described as shorter in large models while reaching CoT-comparable accuracy.

Real-world applications:

• Reducing inference cost and latency by eliciting reasoning with a single-step internal intervention instead of generating long reasoning traces.
• More controllable deployment of reasoning models, including cases where a system prompt or control token (such as no_think) deliberately suppresses reasoning but the task still demands it.
• Interpretability and safety tooling that can suppress or disable reasoning-related computation, useful for auditing or gating model behavior.
• Model diagnostics that use activation of identified features as a signal of whether a task actually engages multi-step reasoning, rather than trusting surface output verbosity.

Industry relevance: the method operates on frozen, inference-only models and steers at a single layer and single generation step, which is compatible with existing serving stacks; but it requires a per-model SAE (pretrained for LLaMA and Gemma, trained from scratch for Qwen), and the paper reports that benefits are limited on tasks such as GPQA that do not rely on multi-step reasoning.

Future Directions

• Broader validation beyond the six models and three benchmarks used here, including tasks, model families, and scales not covered, since cross-dataset feature overlap is only partial (Jaccard@25 of 0.09 to 0.28).
• Clarifying why the GPQA-identified feature fails to improve GPQA itself while improving GSM8K and BBH, and characterizing when steering has little or no effect.
• Moving from behavioral and intervention-level evidence toward fuller mechanistic characterization, since the authors explicitly do not claim the identified features are fully disentangled or isolated reasoning mechanisms.
• Extending the approach beyond single-step or single-feature intervention, and understanding the relationship between feature activation strength and answer correctness, given that stronger activation did not necessarily translate into higher accuracy in the paper's prompt-variation analysis.

Target Audience

Mechanistic interpretability and representation-engineering researchers; practitioners building reasoning systems who care about inference efficiency and controllability; and graduate-level readers comfortable with transformer internals, sparse autoencoders, and activation steering. Readers looking for a beginner introduction to CoT prompting or for a fully mechanistic account of reasoning circuits will find this paper too specialized or too behaviorally focused.

Authors’ abstract

Chain-of-Thought (CoT) prompting often improves the reasoning performance of large language models (LLMs), but the internal signal that triggers this behavior remains poorly understood. Leveraging the sparse features captured by Sparse Autoencoders (SAEs), we propose a systematic framework to analyze and intervene on the internal representations of LLMs, identifying a small set of latent features that are linked to reasoning behavior and can be causally tested through targeted intervention. Across multiple model families and reasoning benchmarks, we show that steering one or a small number of reasoning-related latent features can substantially induce reasoning behavior without explicit CoT prompting, achieving accuracy comparable to CoT. We further show that the identified features are not tied to particular wording patterns or verbosity, and confirm their causal role in reasoning through suppression experiments that impair performance even under CoT prompting. These results suggest that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting. Code is available at https://github.com/Zhenghao-He/LatentCoT.