Does Scaling Toward AGI Depend on What System-1 Lets System-2 Execute?

AI summary

Overview

Research area: Artificial intelligence scaling and AGI strategy, with a specific focus on the division of labor between a lightweight predictor ("System-1") and a resource-intensive executive ("System-2"), spanning LLM reasoning/RL post-training, embodied agents, test-time compute, and agentic tool use.

Technical level: Intermediate overall; the theoretical sections (definitions of the conditional interaction policy, selection-regret bounds, and the Interaction Information Efficiency metric) are Advanced.

Scope: A position/perspective paper, supported by formal definitions and analytical bounds rather than by new experiments, arguing that progress toward AGI should come from co-designing a cheap System-1 that decides which interactions a costly System-2 is allowed to execute, and proposing ρᵤ — the density of utility-relevant interactions — as a measurable coordinate of scaling efficiency.

What This Paper Is About

Progress toward AGI has mostly meant scaling an expensive executive — bigger models, more data and compute, longer reasoning, more agents — even though many of the interactions that executive pays for teach it little. The authors formalize an alternative: a lightweight, semantically aware System-1 predicts the utility of each candidate interaction, an adaptive threshold admits only the promising ones to System-2, and the realized outcomes are fed back to recalibrate System-1. The goal is to show, analytically, when this predict–gate–execute–update loop saves compute, and to argue that the fraction of interactions that actually carry utility (ρᵤ) describes scaling efficiency better than raw compute counts.

Key Contributions

1. The conditional interaction framework. A predict–gate–execute–update loop spanning training, test-time reasoning, adaptive decision-making and agentic tool use, formalized as a conditional interaction policy (Definition 1) that combines an exponential tilt (sharpness β) over predicted utility with a hard utility threshold (κₜ), plus an explicit abstention action when no candidate clears the threshold.

2. The Interaction Information Efficiency (IIE) metric. Utility obtained per unit of system-level cost — including the predictor cost paid on every screened candidate, executed or not — deliberately paired with yield (utility per screened candidate) so that a gate cannot look good merely by starving the learner.

3. Analytical screening results. A proposition showing that, under ideal screening, equal execution costs and a free predictor, the IIE gain over unconditional interaction approaches 1/ρᵤ, with a closed-form generalization to unequal costs, false-positive/false-negative rates and nonzero predictor cost; the loss from imperfect prediction, exploration and distribution shift is characterized, and the authors state that selection regret is bounded by predictor error and the utility margin (the detailed derivation is in a section truncated in the provided content).

4. The ρᵤ scaling hypothesis. The claim that ρᵤ is not a new resource but an efficiency coordinate in a performance law of the form 𝒫 = f(N, D, 𝒞, ρᵤ), that it is produced by learning dynamics and depends on the policy state, and that "informative compute" may be a better description of scaling than total compute.

Main Findings

• Compute savings approach 1/ρᵤ. Under ideal screening and equal execution costs, execution compute falls by a factor approaching 1/ρᵤ, where ρᵤ is the density of utility-relevant interactions (cost-weighted when costs differ). The ideal gain equals 1/ρᵤ exactly when dead and relevant interactions have equal mean cost, and exceeds 1/ρᵤ when dead interactions are more expensive than relevant ones.

• Prediction errors and screening cost erode the gain. With false-positive rate fp, false-negative rate fn, and predictor cost c_pred, the IIE ratio is non-increasing in each of fp, fn and c_pred, and strictly decreasing in any whose coefficient is positive; it recovers the ideal 1/ρᵤ form as fp, c_pred → 0.

• Efficiency and yield must be reported together. False negatives do not directly dilute the IIE ratio (suppressed interactions simply aren't paid for), but they reduce yield by a factor of 1 − fn — so an aggressive gate can raise IIE while starving the learner of signal.

• Diminishing returns of undirected interaction. Under a power law ℒ(𝒞) ∝ 𝒞^(−α), the marginal gain from one more unconditional interaction scales as α·𝒞^(−α−1) and vanishes as compute grows; the authors argue that changing which interactions are executed can shift the curve itself rather than just move along it.

• Wasted rollouts are concrete in RLVR. In Group Relative Policy Optimization with G sampled responses per prompt, when all responses receive the same reward every group-relative advantage is zero, making the prompt degenerate — it yields no relative policy-gradient signal while still consuming G rollouts.

• Pool fraction versus executed fraction. ρᵤ(θₜ) is a property of the candidate distribution that no gate can change; what a gate changes is the executed fraction. Unconditional training makes the two coincide; a perfect gate drives the executed fraction to one.

• The "super parrot hypothesis." A hypothesized three-stage trajectory: imitation-based language learning (Stage 1, roughly 2020–2022), human-followable reasoning via chain-of-thought and RLVR (Stage 2, 2022–2025), and a hypothesized Stage 3 (2026+) of self-coded representations and self-evolution beyond human interpretability — a trend toward "AI for AI" in which the efficiency problem becomes sharper because every self-generated interaction is paid for with the system's own compute.

• Static cascades are not co-design. A static cascade routes once at inference and never receives System-2's outcomes; co-design schedules System-2's compute and storage (rollout budgets, replay buffers, batches, KV caches, context windows) in training as well as inference, and is recalibrated online by realized rewards and gradients. A cascade retrained online on its outcomes lies between the two.

• Policy limits are well behaved. As β → ∞ the conditional interaction policy degenerates to greedy argmax with uniformly broken ties, with an explicit exponential bound on the probability of missing the unique maximizer; when β → 0 it becomes a pure filter that admits without ranking; and a threshold above the maximum predicted utility triggers abstention.

• A hypothesized tighter frontier. The authors frame the design objective as a Pareto frontier over cost, latency and task success, and hypothesize that the co-designed pair dominates scaling System-2 alone — same success more cheaply, or more success at the same budget.

• Explicit claim hierarchy. (i) Premise: future digital and robotic systems may be limited by interaction efficiency more than by parameters or data. (ii) Position: co-design a cheap System-1 with a costly System-2. (iii) Hypothesis: ρᵤ determines how efficiently resources become capability. (iv) Prediction: ρᵤ decays as the system improves, and this decay may account for part of the diminishing returns and of the scaling exponent.

• Stated scope limitation. The principle is confined to situations with a reliable utility predictor and does not substitute for exploration when the predictor is uncertain.

Methodology in Plain English

This is a perspective paper, not an experimental one. The authors set up a precise interface between two components — one that judges, one that executes — and then reason about it mathematically.

They begin by defining what counts as an "interaction" (a generation episode for an LLM, a trajectory for an embodied agent, a tool invocation for an agentic system), noting that these are not physically equivalent but share the property that resources are committed before the outcome is known. They then write down a selection rule: score every candidate with a cheap predictor, admit only those above a threshold, and use the realized outcomes to update the predictor. They split the candidate pool into a utility-relevant part and a "dead" part, and compute the ratio of utility-per-cost with and without the gate — first in an idealized case, then adding false positives, false negatives and screening cost, and finally noting how cost-correlated errors change the accounting.

To make the argument concrete they walk through two regimes: digital training (degenerate RLVR prompts whose advantages are all zero) and physical training (expensive rollouts over a high-dimensional domain-randomization parameter space). They use conceptual diagrams — a roadmap of scaling eras, the "super parrot" trajectory, the cost–latency–success frontier, static cascade versus co-design, and the predict-then-gate loop — to illustrate the argument. The provided content contains no empirical benchmarks, datasets or measured results; the quantitative statements are analytical identities and bounds, and the scaling claims are labeled explicitly as hypotheses and predictions.

Why This Matters

For research, the paper reframes scaling from "how many interactions can we afford" to "which interactions justify their cost," and offers a candidate measurable quantity (ρᵤ, or ρ_info for Bayesian information gain) that could be estimated and tracked rather than assumed. That is a different kind of claim from another architecture or training recipe: it proposes a coordinate along which scaling laws might be rewritten in terms of informative compute.

Real-world applications the authors name or imply:

• RLVR post-training for reasoning models — predicting which prompts will produce non-zero group-relative advantage, so that generation budget is not spent on degenerate prompts.
• Embodied agents and domain randomization — choosing which tasks to evaluate in a high-dimensional simulation parameter space, where rollouts cannot be freely parallelized and consume real time.
• Model routing and adaptive inference — sending each request, task or query only to the System-2 model or amount of thought it actually needs, at training time and at deployment.
• Agentic tool use and computer use — deciding which tool invocations, environment steps and self-proposed experiments in a continual, post-deployment self-evolution loop are worth executing.

Industry relevance is illustrated by the commercial "System One" model Jev (TypeSafe AI, 2026), which returns calibrated probabilities over predefined answers; the authors cite it as evidence of commercial interest in fast, structured judgment models while cautioning that a single product does not establish an industry-wide convergence.

Future Directions

• Measuring ρᵤ empirically. The framework's central hypothesis depends on ρᵤ being estimable in practice, per utility definition — the densities of prompts with non-zero GRPO advantage, of tasks above an adaptation-risk threshold, and of queries whose answers improve with more compute are explicitly not interchangeable.
• Testing the decay prediction. The claim that ρᵤ decays as systems improve, and that this decay accounts for part of the diminishing returns and of the scaling exponent, is stated as a prediction rather than a result and invites empirical falsification.
• Building and co-scaling the predictor. Calibrating System-1 online against System-2's realized outcomes, and resolving the cost of doing so — including the audit executions of suppressed candidates used for refitting, which the paper lists as an amortized cost reported separately.
• Handling unreliable predictors and shift. The authors confine the principle to settings with a reliable utility predictor and state it does not substitute for exploration when the predictor is uncertain; extending the analysis to that regime is left open, as is the treatment of cost-correlated error rates, which the paper recommends reporting as cost-conditioned rates.

Target Audience

Researchers and technical strategists working on scaling laws, inference-time compute, RL post-training for reasoning models, and agentic or embodied systems — particularly those deciding where compute budgets go. The first half is accessible to product and research-strategy readers interested in the System-1/System-2 framing; the formal sections (definitions, IIE accounting, regret bounds) are aimed at readers comfortable with probability, information theory and policy-optimization notation.

Authors’ abstract

Progress toward artificial general intelligence (AGI) has largely relied on scaling a resource-intensive executive, System-2, through larger models, more data and compute, longer reasoning, and more agents. We ask whether the next stage may come instead from co-designing it with System-1, a lightweight, semantically aware predictor that evaluates candidate interactions before execution, so that System-2 acts only on those worth their cost. Our answer is a qualified yes. We study \textit{conditional interaction}, a predict--gate--execute--update loop: System-1 estimates each candidate's utility, System-2 executes those admitted by an adaptive threshold, and the outcomes update System-1. The loop spans training, test-time reasoning, adaptive decision-making, and agentic tool use, with utility and cost defined by the application. Under ideal screening and equal execution costs, execution compute falls by a factor approaching $1/ρ_u$, where $ρ_u$ is the density of utility-relevant interactions (cost-weighted when costs differ); prediction errors, exploration, and distribution shift reduce the saving. We bound selection regret by predictor error and the utility margin. These results motivate our central hypothesis: $ρ_u$ is a measurable coordinate of scaling efficiency and makes informative compute a better description of scaling than total compute.