Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch

AI summary

Overview

Research area: Machine learning — Bayesian optimization (BO), LLM agents, automated machine learning / hyperparameter optimization, and autonomous research agents.

Technical level: Advanced. The paper assumes familiarity with Gaussian process surrogates, acquisition functions, constrained and multi-objective BO, and LLM agent tool-use. The high-level framing is accessible, but the formal sections use metalevel decision-process notation.

Scope (one sentence): The paper proposes "agentic Bayesian optimization," a paradigm in which an LLM agent sits at the center of the optimization loop and decides what to evaluate and how to optimize, while a Bayesian backend (the authors' lenz, built on GPyTorch and BoTorch) supplies calibrated surrogate uncertainty; it is instantiated in an agent called Sara.

What This Paper Is About

Bayesian optimization is sample-efficient but rigid: the surrogate, acquisition function, search bounds, and the split of outcomes into objectives and constraints are all fixed before a run begins, and classical BO cannot easily absorb the informal domain knowledge experts hold in natural language, code, and documentation. Existing LLM-based approaches go to one of two extremes — either the LLM replaces the optimization loop and loses calibrated posterior uncertainty, or it fills a single fixed slot (kernel, surrogate, acquisition selector, candidate proposer) with the overall policy still fixed before the first evaluation. The goal of this paper is to combine both worlds: keep a probabilistic BO backend while letting an LLM agent adapt the optimization strategy mid-run.

Key Contributions

1. The paradigm of agentic Bayesian optimization. An LLM agent deliberates over trial data, surrogate diagnostics, and external context to revise the optimization policy during a campaign, formalized as a metalevel decision process with a state, an action space, and a metalevel policy.

2. A surrogate-augmented autoresearch system. The authors present Sara (a "surrogate-augmented autoresearch agent") and lenz (a highly modular BoTorch-based backend exposing BO primitives through a structured command-line interface), which together instantiate agentic BO. The same architecture is read two ways: from the BO side as agentic BO, and from the autoresearch side as surrogate-augmented autoresearch.

3. An empirical study of agentic BO. Across synthetic and real-world benchmarks, Sara is reported to match state-of-the-art BO performance without prior knowledge, outperform LLM-based BO methods such as LLAMBO, and use natural-language priors to improve beyond Ax.

4. A demonstration in dynamic settings. Sara is shown reconfiguring the full optimization problem on the fly as requirements change — a capability the paper states was not previously available in standard BO.

Main Findings

• Matching standard BO without priors: Sara "preserves the reliability of state-of-the-art BO without prior knowledge" across synthetic and real-world benchmarks. The specific benchmark names, dataset sizes, and numerical results are in the experimental section, which is not included in the provided content.

• Outperforming LLM-based baselines: Sara is reported to outperform LLM-based BO methods such as LLAMBO, and to show "substantially better performance than Centaur" (a hybrid that shares CMA-ES internal state with an LLM through its prompt) in the authors' experiments.

• Natural-language priors add value: Sara uses natural-language priors to "improve beyond Ax," the paper's reference for a mature, standard BO/AutoML framework.

• Mid-run reformulation is possible without restarting: Because lenz stores all outcomes independently and treats the raw trial log as the single source of truth, the agent can add a constraint metric, tighten bounds, switch the objective, or change the objective/constraint partition; the paper states this never invalidates collected data. The illustrative example given is moving from constrained single-objective to multi-objective Pareto-front optimization without restarting.

• Positioning in the landscape (Table 1): Classical BO & AutoML (Ax, TPE, OptFormer) and interactive BO (IBO-HPC, DynaBO) have a calibrated surrogate but no full run-time policy control and no natural-language priors/tool use. Hybrid LLM-BO methods (LILO, BORA, Reasoning BO) have a calibrated surrogate and priors/tool use but no full run-time policy control. LLM-as-optimizer methods (LLAMBO, OPRO, AgentHPO) and open-ended autoresearch agents (HyperAgents) have full run-time control and priors/tool use but no calibrated surrogate. Sara + lenz is listed as the only row with all three.

• Complementarity rather than replacement: The paper states that hybrid LLM-BO methods are complementary to, not superseded by, agentic BO — an LLM-designed kernel, a latent-space generation strategy, or a preference surrogate can each be wrapped as a backend capability the agent invokes when appropriate. The agentic layer's contribution is deciding when and how to combine such capabilities.

• Prompt design matters: The authors report that, beyond the choice of LLM, the design of Sara's system prompt does affect optimization quality; Table 2 lists core directives and their motivation. The table's contents are truncated in the provided content, so its full directive list is not available here.

• Cost is measured but not optimized: Token usage is reported in the paper's Section C.2, but the authors state they do not optimize for computational cost in this work.

Methodology in Plain English

The authors start from a simple observation: a human expert running an optimization campaign does not lock in a strategy at the start — they look at results, shift focus, redefine what counts as a constraint, and change tactics. Standard BO cannot do this, because its policy is fixed before the campaign even though its internal configuration may change according to hand-designed rules.

Their fix is to put an LLM agent in the middle of the loop and give it a well-behaved BO backend to call as a tool. Concretely, the agent can (a) probe the backend with read-only queries such as posterior predictions, acquisition scores, cross-validated R² diagnostics, sensitivity estimates, the incumbent, or the Pareto front; (b) reconfigure the problem by changing search bounds, swapping the acquisition function, or revising which outcomes are objectives and which are constraints; and (c) propose candidate points from the surrogate. Only the agent's final "evaluate" action consumes real experimental budget; all the deliberation in between is free with respect to that budget, though it costs tokens and backend compute. The agent can also stop the campaign.

The backend, lenz, is designed so that reconfiguration is safe: everything is exposed through a command-line interface returning JSON, the trial log is the source of truth, and surrogate models are rebuilt from that log on each query. That means a changed problem definition reuses every prior evaluation instead of restarting. Under the hood it uses Gaussian process surrogates with input normalization and output standardization, optimizes probability of feasibility before switching to constrained logEI once a feasible point is found, and supports expected-hypervolume-improvement acquisition functions for the noisy and deterministic multi-objective cases, with fantasization for batch diversity.

The agent, Sara, is a reasoning LLM given a system prompt that defines her as a surrogate-augmented autoresearch and BO agent, plus reference documentation for the lenz commands. The authors note the agent is deliberately narrower in scope than open-ended scientific-discovery agents, following the general lesson that LLMs are unreliable at direct computation but effective when they delegate to precise tools.

Why This Matters

Impact on research. The paper reframes the LLM-versus-surrogate debate in black-box optimization. Rather than choosing between an LLM that discards posterior uncertainty or a Bayesian loop with a fixed policy, it defines a third position where the agent owns decisions and the surrogate owns calibrated uncertainty. The framework is presented as implementation-agnostic — it prescribes neither the agent's instantiation, nor the surrogate class, nor how context is delivered — and it recovers standard BO and LLM-as-optimizer methods as special cases, which makes it a useful organizing frame for future work.

Real-world applications (drawn from the application areas the paper cites for BO more broadly):

• Hyperparameter tuning of machine learning models, where trials are expensive and natural-language guidance (documentation, prior experiment logs, task descriptions) is abundant.
• Robotics, where evaluating a candidate controller requires a physical trial.
• Experimental design, where each query corresponds to a costly wet-lab or engineering experiment.
• Dynamic requirement changes, where constraints or objectives shift mid-campaign — the paper's demonstration case, in which the agent reframes the problem rather than restarting.

Industry relevance. The work comes out of Meta, and the target setting — expensive black-box tuning with shifting requirements — is common in production ML and engineering pipelines. The architecture is practical in the sense that it wraps existing, permissively licensed tooling (BoTorch, GPyTorch, both MIT-licensed) and exposes them through a single CLI, so an agent can drive established BO routines without replacing them.

Future Directions

• Completing the meta-MDP and training a specialized policy. The paper's Appendix E shows how its formulation can be completed into a full meta-MDP that could form the basis for training or fine-tuning a specialized agentic BO policy; the present work instead relies on a general-purpose LLM without specialized post-training.
• Accounting for computational cost explicitly. The authors report token usage but do not optimize for it, and note that a full meta-MDP may account for these computational costs explicitly.
• Absorbing hybrid LLM-BO capabilities as backend tools. LLM-designed kernels, latent-space generation strategies, and preference surrogates could each be wrapped as backend capabilities the agent invokes when appropriate, with the agent deciding when and how to combine them.
• Evaluating agentic BO inside broader agentic workflows. The paper suggests agentic BO could be readily incorporated into existing agent, AutoML, and end-to-end scientific discovery pipelines.

Target Audience

Researchers and practitioners in Bayesian optimization, AutoML, and hyperparameter tuning; applied scientists and engineers with expensive black-box evaluation budgets; and LLM-agent researchers interested in systems where a language model delegates numerical work to a calibrated tool. Readers focused only on applied results should note that the quantitative benchmark section is not included in the available content — no dataset sizes, benchmark names, or numerical scores from the empirical study are reported here.

Authors’ abstract

Bayesian optimization (BO) has become the standard tool for sample-efficient optimization and owes its efficiency to uncertainty-aware search driven by generic statistical priors. Richer domain priors can improve BO in principle, but encoding them through tailored kernels or problem structure is difficult and rarely done in practice. LLMs can help sidestep this difficulty by making informal priors from natural language, code, and documentation directly available to the optimizer. However, existing LLM-based BO methods either insert the LLM into a fixed role (surrogate, acquisition proxy, or configuration interface) or hand it broad control, sacrificing the systematic exploration that makes BO reliable. We introduce agentic Bayesian optimization: a paradigm in which an LLM agent is the central decision maker in the BO loop while a Bayesian backend provides the uncertainty-aware optimization substrate. The agent configures the problem, queries the backend, selects and commits evaluations, and can revise the optimization strategy during the run by tightening bounds, switching acquisition functions, proposing targeted evaluations, or even reframing the problem following new instructions or observed evidence. We instantiate this idea in Sara, a surrogate-augmented autoresearch agent, and lenz, a modular BoTorch-based backend that the agent can inspect and modify through a structured interface. Across synthetic and real-world benchmarks, Sara preserves the reliability of state-of-the-art BO without prior knowledge, outperforms LLM-based baselines, and uses natural-language priors to improve beyond standard BO. We further demonstrate the practical value of agentic BO in dynamic settings, where Sara reconfigures the full optimization problem on the fly as requirements change, a capability not previously available in standard BO.