Hybrid Reasoning Agents

AI summary

Overview

Research area: Natural language processing / LLM agents — specifically hybrid reasoning that combines continuous "latent" reasoning with text-based tool use.

Technical level: Advanced. The paper assumes familiarity with transformer autoregressive generation, KV caches, supervised fine-tuning, and policy-gradient reinforcement learning (PPO/GRPO-style importance ratios).

Scope: The paper proposes HybridAgent, an agent architecture and post-training pipeline that lets a single LLM alternate between reasoning in continuous latent space and acting through text (tool calls, observations, final answers), and evaluates it at 4B, 8B, and 14B scales on 11 benchmarks.

What This Paper Is About

LLM agents today both reason and interact with the world by generating text token by token, while a separate line of "latent reasoning" work computes internally in continuous hidden states without decoding. Each medium has a drawback: text is slow and verbose, and latent vectors have no native way to call tools and are hard to supervise or audit.

The paper's goal is to close that gap with an agent that "thinks beyond text while still acting through it" — dynamically switching between latent reasoning and text-based interaction inside one unified generation process, and training that agent with dedicated SFT and RL algorithms.

Key Contributions

1. HybridAgent architecture with HybridLink. A lightweight residual adapter maps last-layer hidden states back into the input embedding space, so the model can recurrently generate in latent mode and return to text mode. HybridLink adds roughly 2·d_h² parameters (about 10M parameters for a 4B LLM). Mode entry is triggered by a special <latent> token, and exit is decided when the nearest token embedding to HybridLink's output is </latent> (by cosine similarity). Because latent steps can be mapped to their nearest token embeddings, latent thoughts are rendered as readable "pseudo-text" and are directly inspectable.

2. Multi-turn hybrid generation over continuous working memory. Instead of re-tokenizing prior turns, the agent accumulates key-value (KV) entries from text, latent thoughts, and tool observations into a persistent working memory that is carried across turns, preserving the full hybrid context.

3. A dedicated post-training pipeline. HybridSFT stabilizes tool-integrated hybrid generation through format activation, HybridLink activation, and a two-pass self-conditioned training scheme that replaces ground-truth latent embeddings with the model's own HybridLink outputs. HybridRL then scales reasoning via hybrid policy factorization (Proposition 4.1) and exact rollout replay, with a reward of r_correct + r_format + r_tool and no explicit reward for mode usage.

4. Empirical evaluation at three model scales on 11 benchmarks. Hybr

Authors’ abstract

How can we build an agent that thinks beyond human language, yet still acts through it in the external environment? Modern LLM agents still reason and interact through text, while latent reasoning offers an efficient internal medium but lacks native tool access and reasoning transparency. We bridge these two directions with Hybrid Reasoning Agents (HybridAgents), which dynamically navigate between latent reasoning and text-based interaction throughout a unified generation process. Architecturally, a lightweight HybridLink enables seamless transitions between text and latent modes, while the agent's continuous working memory preserves lossless hybrid context across multiple turns. These designs preserve tool-integrated agentic capabilities while making latent thoughts directly inspectable. Building on this architecture, we further develop dedicated post-training algorithms for hybrid reasoning: HybridSFT stabilizes tool-integrated hybrid generation by teaching the backbone to coordinate with HybridLink, then HybridRL scales reasoning by learning when to think in latent space and when to act through text. We evaluate HybridAgent at 3 model scales across 11 benchmarks covering mathematics, science, medicine, search, and agentic reasoning. Compared with frontier LLM Agents and agentic frameworks spanning both text-based and latent methods, HybridAgent consistently improves performance by 10.4%, while delivering 35.5%-54.7% token reduction, 1.8x-2.6x faster inference, and 2.7x-3.7x cost savings. Code and data are available at https://github.com/togethercomputer/HybridAgent.