Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models

AI summary

Overview

Research area: Mechanistic interpretability of large language models, specifically how LLMs internally compose multiple facts during multi-hop reasoning.

Technical level: Intermediate (the paper assumes familiarity with transformer components such as MLP layers, attention layers, hidden states, and probing techniques).

Scope: The paper challenges the "hop-aligned circuit hypothesis" for latent multi-hop reasoning by documenting a phenomenon called layer-order inversion, and proposes a probabilistic recall-and-extract framework to explain it.

What This Paper Is About

Recent work claims that LLMs solve multi-hop questions by resolving one hop at a time across layers — bridge entities are computed in shallow layers and passed forward before the final answer entity appears. The authors test this claim on real-world multi-hop queries and find systematic counterexamples: later-hop answer entities can become decodable earlier than the bridge entities they supposedly depend on. The paper's goal is to document this layer-order inversion, explain it with a probabilistic framework, and use that framework to reinterpret both earlier circuit findings and multi-hop failures.

Key Contributions

1. The paper reports layer-order inversion for the first time, showing that later-hop entities can become decodable earlier than bridge entities in LLMs, and that this effect strengthens with hop count — evidence that hop-aligned circuits do not generalize to higher-hop queries.
2. It introduces a probabilistic recall-and-extract framework that models multi-hop reasoning as probabilistic knowledge recall (vertical across layers, horizontal across tokens) followed by selective extraction, with shallow recall of deeper-hop knowledge acting as an intuition-like signal.
3. It shows through probing and causal intervention that this last-token intuition-like recall can influence answer selection, with stronger effects in weaker models and higher-hop queries.
4. It uses the framework to account for prior empirical phenomena, including hop-aligned circuit observations and shortcuts, and to provide a mechanistic diagnosis of multi-hop failures even when all relevant single-hop facts are answered correctly.

Main Findings

• Layer-order inversion exists and scales with hops: On MQuAKE, the final-hop entity at the last token often becomes decodable as early as, or earlier than, bridge entities at the subject token. For example, on 4-hop queries in Llama 3, the final entity e4 at the last token emerges earlier than the first bridge entity e1 at the subject across all settings.
• The effect is absent at two hops: The authors observe no layer-order inversion in two-hop queries, while the effect becomes more pronounced as hop count increases. They attribute prior hop-aligned conclusions partly to the two-hop and synthetic, template-based scope of earlier analyses.
• Latent reasoning is still present: When probing the subject token, all entities are frequently decodable, with the first bridge entity e1 often achieving the highest decoding rate. Under filtered settings, earliest decodable layers tend to shift deeper as hop index increases, consistent with progressive information accumulation.
• Last token behaves differently from subject token: At the last token, bridge entities remain decodable but the subject entity e0 is rarely decoded, and the final-hop entity is decoded with higher probability than earlier-hop entities.
• Recall is both vertical and horizontal: Vertical recall across layers has been suggested before, but the authors argue it cannot alone explain layer-order inversion. They propose horizontal recall across tokens, where accumulated context lets deeper-hop knowledge become available without explicitly decoding all preceding bridge entities.
• Last-token shallow recall acts as an intuition-like signal: For the weaker GPT-J model, intervening on shallow activations at the last tokens produces stronger effects than interventions at the subject token, and the effect increases in higher-hop queries. For the stronger Llama3-8B model, last-token interventions are much weaker than subject-token interventions.
• Attention extracts answer-relevant knowledge: Answer-entity decodability at the last token rises sharply in deeper layers, and perturbing deep-layer attention at the last token has clear effects that grow with hop count.
• Circuit-like patterns are reinterpreted as probabilistic: Hop-aligned observations are recast as the most probable recall trajectory across token positions (e.g., the "e1 to e2" trajectory at the subject position), which accommodates higher-hop behavior that strict hop alignment cannot.
• Shortcuts are reinterpreted: Shortcuts are viewed as cases where the intuition-like signal is used directly for answer selection, with these direct recall paths becoming more influential for weaker models or harder queries.
• Two causes of multi-hop failure: On the Incorrect subset, where all constituent single-hop facts are answered correctly, the authors identify (RQ3.1) insufficient shallow recall of deeper-hop knowledge, especially at the last token, and (RQ3.2) insufficient knowledge extraction in deeper attention layers, where the final-hop entity often fails to dominate the decoding distribution.
• Missing cases: When single-hop facts are answered incorrectly, early-hop knowledge such as the first-hop entity also exhibits low recall probability under the multi-hop query, preventing correct answer generation.
• Hidden-state similarity gaps widen with hop depth: Comparisons between multi-hop queries and their corresponding single-hop queries show the gap between Correct and Incorrect cases increases as hop depth grows, even though the same underlying single-hop knowledge is answered correctly.

Methodology in Plain English

The authors evaluate two decoder-only language models, GPT-J-6B and Llama 3-8B, on MQuAKE, a benchmark containing multi-hop questions with up to four hops, multiple natural-language verbalizations per query, and explicit annotations of intermediate facts. They use greedy decoding (temperature set to 0) for deterministic inference.

They split MQuAKE by model predictions into three subsets: Correct (both multi-hop and all single-hop questions answered correctly; 408 cases for GPT-J and 1,214 for Llama3-8B), Incorrect (only the multi-hop question fails; 842 for GPT-J and 1,294 for Llama3-8B), and Missing (the multi-hop question and at least one associated single-hop question fail; 1,627 for GPT-J and 920 for Llama3-8B).

The main tool is Patchscopes, an unsupervised probing method that extracts a hidden state from a chosen token and layer of a source query, patches it into a placeholder token in a target prompt, and reads out what knowledge the generation reveals. The authors probe the last subject token and the last token of the query, repeat the patching procedure three times per query, and report both raw and filtered outputs. Because shallow-layer generations can be noisy, they apply global and local similarity-based filtering, focusing on 90% global filtering and 90% local filtering as the strongest settings.

To test causality, they design a three-run intervention: a Clean Run on the original query, a Corrupted Run where Gaussian noise is added to the subject embedding, and a Patched Run where an activation at a given token position and layer in the clean run is replaced with its corrupted counterpart. They measure the intervention effect as the drop in the probability of the correct answer. They apply this at the subject token and the last 5 tokens across hidden states, MLP outputs, and attention outputs.

A supplementary hidden-state similarity analysis compares attention projection outputs, MLP inputs, and MLP outputs between a multi-hop query and its corresponding single-hop queries. Since cross-layer similarities turn out to be substantially weaker than same-layer similarities, they restrict comparisons to the same layer.

Failure analysis applies Patchscopes to the Incorrect and Missing subsets and compares layer-wise hidden-state similarity between Correct and Incorrect cases.

Why This Matters

Impact on research: The paper directly challenges a widely used interpretive frame in mechanistic interpretability — that LLMs decompose multi-hop queries into sequential, layer-aligned hops. It offers an alternative probabilistic account that is claimed to subsume hop-aligned observations as high-probability trajectories rather than deterministic structure, and it reframes multi-hop failure as a recall-extraction problem rather than purely a knowledge problem. It also raises practical implications for knowledge editing, arguing that key representations should not be derived only from subject-centric samples, and that attention-encoded knowledge deserves more study since existing editing methods mainly focus on MLP-stored knowledge.

Real-world applications (implied by the paper's framing):

• Knowledge-intensive question answering systems that must integrate several facts, where the paper's diagnosis of insufficient recall versus insufficient extraction could guide debugging.
• Knowledge editing and model intervention, since the paper argues representations keyed only on subject entities may not transfer across multi-hop and single-hop contexts.
• Model selection and deployment for reasoning workloads, given the finding that intuition-like last-token recall has stronger effects in the weaker model examined.
• Error attribution in production pipelines, where a model answers each single-hop fact correctly but still fails the composed query.

Industry relevance: Teams building retrieval-augmented or knowledge-grounded assistants, and teams doing model editing or interpretability audits, can use this framework as a diagnostic lens. The paper also provides a released code repository at https://github.com/laquabe/Layer-Order-Inversion.

Future Directions

• Extending the analysis beyond the relatively small models tested; the authors state they were unable to evaluate larger models (e.g., 70B parameters) due to computational constraints, and explicitly avoid claiming universality across model scales.
• Verifying how intuition-like recall signals can be guided, strengthened, or incorporated into inference strategies — the authors state they do not yet verify this and leave it for future work.
• Further theoretical analysis and controlled experiments on how intuition-like recall interacts with step-by-step reasoning under the probabilistic recall framework.
• Investigating how intuition-like recall can be strengthened to improve LLM reasoning performance, which the conclusion names as the intended future direction.

Target Audience

This paper is best suited to mechanistic interpretability researchers, NLP graduate students, and practitioners working on multi-hop question answering, knowledge editing, or model diagnostics. It will be most valuable to readers already comfortable with probing hidden states, causal tracing, and the MLP-versus-attention distinction in transformers; readers without that background can still follow the high-level argument about layer-order inversion, but the causal intervention and hidden-state similarity sections require intermediate technical grounding.

Authors’ abstract

Large language models (LLMs) perform well on multi-hop reasoning, yet how they internally compose multiple facts remains unclear. Recent work proposes \emph{hop-aligned circuit hypothesis}, suggesting that bridge entities are computed sequentially across layers before later-hop answers. Through systematic analyses on real-world multi-hop queries, we show that this hop-aligned assumption does not generalize: later-hop answer entities can become decodable earlier than bridge entities, a phenomenon we call \emph{layer-order inversion}, which strengthens with total hops. To explain this behavior, we propose a \emph{probabilistic recall-and-extract} framework that models multi-hop reasoning as broad probabilistic recall in shallow MLP layers followed by selective extraction in deeper attention layers. This framework is empirically validated through systematic probing analyses, reinterpreting prior layer-wise decoding evidence, explaining chain-of-thought gains, and providing a mechanistic diagnosis of multi-hop failures despite correct single-hop knowledge. Code is available at https://github.com/laquabe/Layer-Order-Inversion.