ImprovEvolve: Ask AlphaEvolve to Improve the Input Solution and Then Improvise

AI summary

Overview

• Research area: LLM-guided evolutionary computation (genetic programming) applied to mathematical optimization and automated algorithm discovery. arXiv category cs.NE.
• Technical level: Intermediate. Readers need some familiarity with evolutionary search, local optimization, and the idea of using LLMs to write code, but the paper's central idea is explained conceptually.
• Scope: The paper proposes a reformulation of what an LLM is asked to evolve — a set of three modular optimization operators (generate_config, improve, perturb) driven by basin-hopping — and evaluates it on three mathematical optimization benchmarks, reporting new best-known results on all three.

What This Paper Is About

The paper addresses a practical obstacle in LLM-driven mathematical discovery: asking a model to write a complete, end-to-end optimizer in a single program is hard, and the resulting code often relies on generic routines such as L-BFGS-B that cope poorly with rugged, constrained landscapes. The authors' goal is to change what the LLM is asked to produce — not an end-to-end solver, but three narrowly scoped subroutines for initialization, local improvement, and perturbation — and to let classical basin-hopping machinery handle the global search. The resulting program is then evaluated on hexagon packing, the second autocorrelation inequality (ACI 2), and spherical codes.

Key Contributions

1. A reformulation of the LLM's task. Instead of evolving a monolithic program that directly outputs a candidate solution (as in AlphaEvolve), ImprovEvolve evolves a class implementing three methods: generate_config(seed) for feasible initialization, improve(x) for local refinement, and perturb(x, σ) for intensity-controlled exploration.
2. A basin-hopping validation scheme with evolved subroutines. The evolved class is executed via a two-stage algorithm: Stage A samples K initial configurations (default K = 10) and refines each with improve, keeping the best; Stage B performs R rounds of perturb-and-improve with a geometrically decaying perturbation intensity from σ_max = 100 to σ_min = 0.001 over M = 10 iterations, using monotonic acceptance (temperature T = 0).
3. New state-of-the-art results on three benchmarks. New packings for hexagon-in-hexagon for n = 11, 12, 15, and 16, and additionally for n = 14, 17, and 23 after minimal expert tuning; a raised lower bound for the second autocorrelation inequality from 0.96102 to 0.96258; and lower maximum cosines on the majority of 90 catalogue spherical codes.
4. An honest ablation and a demonstration that human tuning still matters. The paper shows both validation stages are essential, that a version edited by a human expert (ImprovEvolve+E) outperforms the pure evolved program on some tasks, and that asking an LLM to make the same edits autonomously produced plausible-sounding but empirically harmful changes.

Main Findings

• Hexagon packing, n = 11: ImprovEvolve reaches an enclosing side length of 3.9245, against 3.9301 for AlphaEvolve, 3.9296 for the GigaEvo baseline, 3.9249 for the reported global-solver results, 3.9379 for CodeEvolve, and 3.9434 for the previously best human-known value.
• Two structurally distinct optima were found for n = 11. One packing (L = 3.9269) is structurally different from AlphaEvolve's; another (L = 3.9245) resembles AlphaEvolve's structural basin but reaches a deeper optimum. The authors read this as evidence of multiple deep basins and as motivation for Stage A diversity.
• Generalisation to unseen problem sizes. A program evolved only for n = 11 was validated for all 11 ≤ n ≤ 24 with K = 100 seeds, R = 100 rounds, and a σ schedule of np.geomspace(1000, 0.001, 25), producing new state-of-the-art packings for n = 12, 15, and 16 — without re-evolution. The GigaEvo baseline, by contrast, produced overlap errors when its n = 11 program was validated on n > 11.
• Second autocorrelation inequality: The evolved program reaches C(f) = 0.9512, below AlphaEvolve's 0.96102 but above the GigaEvo baseline's 0.9478, ThetaEvolve's 0.94690, and CodeEvolve's 0.88110. Known bounds are 0.88922 ≤ C₂ ≤ 1.
• ImprovEvolve+E pushes ACI 2 to 0.96258. Starting from AlphaEvolve's 50,000-step solution and applying three human edits — a progressive resolution schedule scaling up to 1.6 million steps, removal of the 10⁻¹⁰ lower clipping, and a raised L-BFGS iteration cap — the improve–perturb loop reaches C(f) = 0.96258 in just fifteen iterations.
• Statistically significant advantage over the baseline. Across three independent 100-generation runs, HEX 11 improves from a GigaEvo mean of 3.9309 ± 0.0016 to 3.9261 ± 0.0014 (Welch's t = −3.96, p = 0.0086), and ACI 2 from 0.9430 ± 0.0041 to 0.9494 ± 0.0019 (t = 2.43, p = 0.0493).
• Spherical codes: On 90 catalogue codes spanning d ∈ [8, 16] and N ∈ [26, 1021], the calibrated general improver ImprovEvolve++ strictly lowers the maximum cosine on 53 of them, a mean relative improvement of 0.1765% (scoring non-improvements as zero) — 2.35× the best prior program (ImprovEvolve 0.0752%, ImprovEvolve+E 0.0618%). The largest gains occur at d = 13 and d = 16.
• Diagnostic spherical-code configuration (N, d) = (296, 16): The maximum cosine falls from 0.30606 (Cohn catalogue) to 0.30591 (ImprovEvolve), 0.30497 (ImprovEvolve+E), and 0.29865 (ImprovEvolve++).
• Scale context for the spherical-code gains: The best-known 11D configurations at N = 593 (the state-of-the-art 11D kissing number) and N = 596 differ in maximum cosine by only 0.18%, so fractional-percentage improvements correspond to substantial structural rearrangements.
• Ablation on HEX 11: Stage A only (K = 1000, R = 0) gives 3.9282; Stage B only (K = 1, R = 100) gives 3.9269; A + B (K = 100, R = 5) gives 3.9245; the LLM-edited variant gives 3.9270; a hybrid using the LLM-edited generate_config gives 3.9310.
• Monotonic acceptance (T = 0) was preferred over higher-temperature basin-hopping. The authors argue high temperatures prevent full refinement, low temperatures barely help escape local optima, and T > 0 risks discarding configurations on the path to the global optimum (illustrated by the L = 4.6188 arrangement for HEX 17).
• LLM self-tuning failed where humans succeeded. Prompted to make minimal non-structural edits, Gemini 3 Pro increased Langevin noise for the GigaEvo baseline (the noise should have been decreased, yielding overlapping hexagons) and, for ImprovEvolve, switched the lattice orientation from point- to flat-topped, which helped dense packings at n = 13, 21, 31 but suppressed the stochastic exploration Stage A relies on.
• Computational cost: A full hexagon-packing run takes roughly 10 hours on a single machine; the second autocorrelation inequality takes roughly 40 hours because each improve call performs high-dimensional L-BFGS optimization over step functions with up to tens of thousands of parameters.
• Hexagon packing beyond reported results: ImprovEvolve+E was also evaluated for 25 ≤ n ≤ 30, where no packings had previously been reported, and produced structured, non-chaotic packings.

Methodology in Plain English

The authors keep the outer evolutionary loop from prior work — they build on the open-source GigaEvo framework, using a single-island MAP-Elites archive with fitness as the only behavior descriptor, discretized into 150 bins, and GigaEvo's default mutation pipeline (InsightsStage, LineageInsights, and aggregate statistics) under common settings of N_elites = 6, N_parents = 2, and N_offspring = 10, all on CPU.

What changes is the interface of the program being evolved. Rather than one function that emits a final answer, each candidate program must implement three methods. generate_config turns a random seed into a feasible starting configuration. improve takes a configuration and returns one with fitness at least as good. perturb randomly explores the space at an intensity σ.

Each candidate program is then scored by running it inside a fixed, human-written search procedure. Stage A generates K configurations with distinct seeds, refines each with improve, and keeps the best as the incumbent. Stage B repeatedly perturbs the incumbent with a σ that decays geometrically from a large value to a small one, refines each perturbed point with improve, and accepts the result only if fitness strictly improves. The schedule restarts on every round, so the search alternates between attempted large jumps and fine local refinement. The program's fitness is the best solution quality observed during that run.

For initialization, Gemini 3 Pro generates N = 5 starting programs and refines them in the same chat to fix validator errors, after which the faster Gemini 3 Flash Preview runs the evolution — a two-tier choice made because running everything with a top-tier model is prohibitively expensive. Because the evolution problem is stated over a class with a fixed interface rather than a fixed answer, the best evolved program can later be pointed at any starting point, including a construction produced by a human expert or by a competing method. That is exactly how the ACI 2 and spherical-code improvements were obtained: the evolved operator was resumed from AlphaEvolve's existing solution rather than run from scratch.

Why This Matters

The paper's central claim is methodological rather than purely numerical: the difficulty of LLM-driven mathematical discovery may lie less in the model's mathematical reasoning than in the shape of the question it is asked. Decomposing a problem into initialization, local improvement, and exploration — and handing the global search policy to a classical algorithm — makes the code the model must write smaller, more interpretable, and easier to debug, and it makes the evolved artifact reusable on inputs the model never saw during evolution.

Real-world applications of the problem classes studied here:

• Packing and layout: The hexagon-packing formulation is a constrained geometric packing problem, the same family as container loading, warehouse slotting, and cutting-stock planning in logistics and manufacturing.
• Sphere packing and spherical codes: These arise in coding theory, signal constellation design and antenna placement in communications, and in structural chemistry and molecular modelling, where point arrangements on a sphere approximate physical configurations.
• Analysis and signal processing: Bounds on autoconvolution inequalities are extremal-function problems in harmonic analysis; tighter bounds sharpen statements about how mass concentrates under convolution.
• Automated algorithm design more generally: The improve / perturb interface is problem-agnostic, so the pattern could be reused wherever a costly global optimizer sits on top of a problem-specific local heuristic.

Industry relevance is mainly for groups already applying LLM agents to combinatorial optimization, scientific computing, or automated solver configuration: the paper reports that with lightweight human edits, ImprovEvolve matches or exceeds results previously obtained by formalized global solvers such as FICO Xpress and SCIP from the opposite direction. The impact statement also flags the ethical consideration of computational cost — each evolutionary run consumes significant energy and API resources — which the authors mitigate with their two-tier model strategy and modular framework.

Future Directions

• Broader generalisability. The limitations section states that the main quantitative evaluation is confined to three mathematical optimization problems, so applicability to other problem classes remains to be demonstrated.
• Reducing reliance on single-run headline numbers. The paper notes that, apart from the multi-run study, the state-of-the-art figures are single-run best results and may vary given the stochastic nature of both evolution and basin-hopping.
• Making LLM self-tuning trustworthy. The experiments where an LLM edited constants and arguments produced harmful changes, suggesting a need for agentic loops that evaluate candidate edits under different configurations rather than proposing them blind.
• Tailored decompositions and agentic workflows. The authors propose combining human-designed module decompositions with agents that decide autonomously which modules to refine and when to invoke external solvers — motivated by the observation that reproducing the 592-sphere 11D kissing number construction took roughly 20 chat exchanges with Gemini 3 Pro given the LaTeX source of the original article, with intermediate versions yielding fewer than 100 valid spheres.

Target Audience

Researchers and practitioners in LLM-guided evolutionary computation, automated algorithm discovery, and global optimization. It is also relevant to applied mathematicians working on packing, extremal analysis, or spherical-code problems who want to know how far LLM-driven search now reaches on their benchmarks, and to engineers building AI-assisted scientific computing tools who care about how a task should be decomposed before it is handed to a model. The design of the generate_config / improve / perturb interface and the failure analysis of LLM self-editing are the parts most transferable outside the three benchmarks studied.

A note on naming: the arXiv listing gives the title as "ImprovEvolve: Ask AlphaEvolve to Improve the Input Solution and Then Improvise," while the paper's own title page and running header read "ImprovEvolve: Basin-Hopping Meets LLM-Guided Evolutionary Search."

Authors’ abstract

Recent advances in LLM-guided evolutionary computation, particularly AlphaEvolve, have demonstrated remarkable success in discovering novel mathematical constructions and solving challenging optimization problems. In this article, we present ImprovEvolve, a simple yet effective technique for enhancing LLM-based evolutionary approaches such as AlphaEvolve. Given an optimization problem, the standard approach is to evolve program code that, when executed, produces a solution close to the optimum. We propose an alternative program parameterization that maintains the ability to construct optimal solutions while reducing the cognitive load on the LLM. Specifically, we evolve a program (implementing, e.g., a Python class with a prescribed interface) that provides the following functionality: (1) propose a valid initial solution, (2) improve any given solution in terms of fitness, and (3) perturb a solution with a specified intensity. The optimum can then be approached by iteratively applying improve() and perturb() with a scheduled intensity. We evaluate ImprovEvolve on challenging problems from the AlphaEvolve paper: hexagon packing in a hexagon and the second autocorrelation inequality. For hexagon packing, the evolved program achieves new state-of-the-art results for 11, 12, 15, and 16 hexagons; a lightly human-edited variant further improves results for 14, 17, and 23 hexagons. For the second autocorrelation inequality, the human-edited program achieves a new state-of-the-art lower bound of 0.96258, improving upon AlphaEvolve's 0.96102.