Raven: The Harness of Harnesses for Composable Agentic Intelligence

AI summary

Overview

Research area: Multi-agent AI systems, specifically the automated construction, evolution, and composition of model-specific "harnesses" (the tool interfaces, context management, skills, execution policies, and recovery mechanisms wrapped around a language model).

Technical level: Advanced. The paper pairs a systems architecture with a measure-theoretic formalization of capability, including Hoare-style assertions, conditional outcome kernels, and probability bounds on composed reliability.

One-sentence scope: Raven is an open-source multi-agent ecosystem in which a Host Agent decomposes goals into a DAG of subtasks matched to specialized model–harness pairs, with modular harness evolution, persistent memory, and reusable skills, backed by theory on when such composition reliably solves tasks that no individual agent in the pool can.

What This Paper Is About

As LLMs improve, agents are moving from isolated, domain-specific tasks toward long-horizon, cross-domain workflows. This exposes two problems: manually engineering a harness for every model and domain does not scale, and tightly coupling a harness to one domain limits its generality. Raven's goal is to shift the question from "how do we engineer a stronger harness for one domain" to "how do we autonomously construct specialized harnesses, improve them through experience, and orchestrate them across domains."

Key Contributions

The paper states three main contributions:

1. An open ecosystem for harness composition. An architecture that coordinates native and third-party agents through a shared interface and explicit execution dependencies, with modular harness evolution, persistent memory, and skill reuse (Sections 3 to 6).
2. A theory of composable agentic intelligence. A formalization of task-relative capabilities and sufficient conditions for reliable composition and expanded task coverage under a shared resource budget (Section 2).
3. A benchmark and evaluation across capability levels. The Multi-Agent Orchestration Benchmark (MAOB) for specialist selection and dependency prediction, plus empirical evidence organized across orchestration, the four native specialists, harness adaptation, and skill reuse (Section 7).

Main Findings

• Orchestration quality (MAOB): Raven ranks first among the compared systems on all four graph metrics under both tested backbones, with Exact Match gains of 10.4 and 10.5 percentage points over the strongest baseline for the two backbones, respectively. MAOB measures specialist selection and dependency prediction by comparing proposed DAGs against reference graphs before any worker execution.
• Native specialists: Raven ships four native specialists — Raven-Research, Raven-Code, Raven-Design, and Raven-Oncall — alongside independently developed agents including Claude Code, Codex, Hermes Agent, and OpenClaw, connected through execution adapters. Domain-specific evaluations characterize the four native specialists; specific numerical results are not reported in the available text.
• Harness evolution: Modular harnesses expose execution policies that an external evolver can modify within defined boundaries, building on the authors' prior HarnessBank work. The published HarnessBank experiments assess the evolution method with a frozen backbone.
• Skill reuse: The published SkillCorpus experiments show that a curated skill library improves Raven on three benchmarks, with larger gains than OpenClaw on two of them. Skill Forge retrieves procedures from local skills, memory-derived skills, and SkillHub, whose corpus and retrieval design build on SkillCorpus.
• Deterministic soundness (Lemma 1): For a compatible plan starting from admitted initial state–ledger pairs, success of the first j operations establishes the j-th invariant and enables the next operation's precondition; if every operation succeeds and the resource-feasibility inequality holds, the protocol delivers a correct record within budget B and terminates.
• Reliability bound (Theorem 1): Conditioning on a valid planning pair, the conditional success probability satisfies q_t(ζ, π) ≥ [1 − ε(ζ, π)]₊, where ε(ζ, π) is the sum of per-operation error bounds over all nodes and edges. No independence assumption is required.
• Host-level reliability (Proposition 1): If the host produces a valid planning pair with probability at least 1 − η_H(t, B) and per-plan error is uniformly bounded by ε̄(t, B), then p_{S_H}(t, B) ≥ (1 − η_H(t, B))(1 − ε̄(t, B)).
• Capability beyond individual agents (Corollary 1): If the composed lower bound is at least 1 − δ while every individual agent's p_a(t, B) is strictly below 1 − δ, then the task lies in the system's new-task coverage set and outside every individual agent's capability set.
• Explicit complementary construction: With two independent fair bits U and W, an agent that can read only U, an agent that can read only W, and a synthesis agent with neither interface each achieve success probability at most 1/2 on a parity task, while a three-node plan with exact-copy handoffs achieves p_{S_H}(t, B) = 1 under a sufficient budget. The construction applies for δ < 1/2. The paper explicitly notes it does not assert superiority over a single agent equipped with both read interfaces.
• Stated limitation of the theory: The results are sufficient conditions for reliable composition, and their practical applicability depends on whether the implemented agents and host actually satisfy them.

Methodology in Plain English

The authors build a system and then formalize it. On the system side, each executable model–harness pair is treated as a unit of composition. A Host Agent matches subtasks to registered capabilities, represents dependencies as a directed acyclic graph, validates the plan before dispatch, schedules ready nodes, records artifacts for handoff and reuse, handles exceptions, and integrates deliverables. A host archive retains user context, and the optional EverOS backend provides semantic access to past experience. Skill Forge supplies reusable procedures.

On the theory side, they define a task as a request plus an input law, an environment, and a correctness judge; define capability as the probability of producing a correct record within a total cost budget; define a plan as a typed DAG with contracts; and prove that if each operation's contract is realized with bounded error, the composed system inherits a lower bound on success. They then define decomposition-level complementarity (each node has at least one eligible executor, but no single agent is eligible for every node) and show when the composition strictly exceeds individual agents.

Empirically, they separate the evaluation into four parts — planning quality, specialist execution, harness adaptation, and skill reuse — and introduce MAOB to isolate planning by scoring proposed DAGs against reference graphs before execution. The evolution and skill experiments are drawn from the previously published HarnessBank and SkillCorpus work, using a frozen backbone for evolution.

Why This Matters

Impact on research: The paper reframes agent improvement as a composition and ecosystem problem rather than a single-harness engineering problem, and it supplies a formal account of when composition adds capability rather than just cost. Treating capability as task-relative, budget-bounded coverage gives a common yardstick for comparing individual agents, composed systems, and the host controller. MAOB also introduces an evaluation that isolates planning from execution.

Real-world applications (drawn from the paper's own examples):

• Cross-domain product development: the paper's illustrative FPS game project combines research, gameplay programming, visual design, integration testing, and sustained operation across specialists coordinated by the host.
• Research workflows, via the Raven-Research specialist.
• Software engineering and coding, via Raven-Code and third-party coding agents such as Claude Code and Codex.
• Visual design, via Raven-Design, and on-call/operations, via Raven-Oncall.

Industry relevance: Raven is released as open-source software with a public repository, and the architecture is explicitly designed to absorb additional third-party agents through execution adapters and registry entries while preserving their tools and internal execution policies. That makes it a candidate integration layer for organizations that already run heterogeneous agents rather than a replacement for them.

Future Directions

• Quantifying recovery gains: The paper notes that recovery on failed paths preserves the reliability lower bound if the first plan's all-success path still delivers within budget, but that quantifying additional recovery gains requires accounting for later plans and reached states.
• Decidability and runtime verification: Validity is described as an analytical property of a planning pair that need not be decidable by the runtime; how to make compatibility and validity practically checkable is left open.
• Broadening the agent pool: The registry is designed to accommodate agents as they are integrated, raising the question of how capability profiling should be done for arbitrary new agents and whether the complementarity conditions hold at scale.
• Extending the empirical record: The provided text characterizes the four native specialists but does not report their numerical results, and the evolution and skill findings are drawn from prior published experiments, leaving room for unified end-to-end evaluation.

Target Audience

Researchers and engineers working on multi-agent systems, agent harness design, and orchestration; practitioners building cross-domain agent platforms who need to integrate heterogeneous third-party agents; and theory-oriented readers interested in formal conditions for when composition outperforms individual agents. A background in probability and formal verification helps with Section 2, while Sections 3 to 7 are aimed at system builders.

Authors’ abstract

As large language models advance, AI agents are moving beyond isolated, domain-specific tasks toward long-horizon, cross-domain workflows. This transition exposes two challenges: increasing harness complexity makes manual design difficult to scale, while tighter coupling to specific domains limits the generality of a single harness. The central question thus shifts from how to engineer a stronger harness for one domain to how to autonomously construct specialized harnesses, improve them through experience, and orchestrate them across domains. We introduce Raven, \emph{The Harness of Harnesses}, an open-source multi-agent ecosystem that automatically constructs and evolves modular harnesses for specific models and domains, treating each executable model--harness pair as a composable unit of intelligence. To support an \emph{All-Domain Collaboration Network}, its Host Agent decomposes goals, matches subtasks to specialized agents, coordinates execution dependencies, and integrates results, while a host archive and EverOS preserve experience across tasks and Skill Forge makes that experience available as reusable procedures. Our theory establishes sufficient conditions for such composition to expand reliable task coverage beyond that of the available individual agents under a shared resource budget. On complex and long-horizon tasks, Raven significantly outperforms the state-of-the-art agent systems, pushing the frontier of composable agentic intelligence.