GEPA-Curator: Curated Learning for Self Improving Agents

AI summary

Overview

Research area: Natural language processing, specifically LLM agent self-improvement and reflective prompt optimization.

Technical level: Intermediate. The paper assumes familiarity with LLM agents, tool-calling harnesses, and the idea of using natural-language feedback in place of gradients, but the core argument is intuitive.

Scope: The paper introduces GEPA-Curator, a wrapper ("Curator") around existing reflective prompt optimizers that decides which agentic task experiences should drive prompt updates, and evaluates it on WebArena/GoBrowse, ALFWorld, AppWorld, and τ-bench with Gemini 3.1 Pro and Claude Opus 5 as optimizer models.

What This Paper Is About

Reflective prompt optimizers like GEPA improve an agent's prompt by looking at failed rollouts and having a "Reflector" LLM diagnose what went wrong and rewrite the prompt. The problem is that an agentic failure does not necessarily implicate the prompt: it can come from an unavailable tool, a harness limitation, stochastic execution, a noisy LLM-as-a-judge, or a task that is simply impossible or ambiguous. The paper argues that feeding such failures to the optimizer can produce useless or actively harmful prompt mutations, and proposes curating both the tasks and the historical context given to the optimizer.

Key Contributions

1. Task curation: The Curator inspects accumulated per-task execution history and decides, each optimization step, which training tasks to run and which to temporarily skip, filtering out tasks likely to yield noisy or unreliable learning signal (repeated stable success, failures caused by harness/environment limits, inconsistent evaluations of similar outcomes).
2. Prior success retrieval: For tasks in the current minibatch, the Curator retrieves one past successful trajectory for the same task as a trusted reference, so the Reflector can contrast a current failure against a known success rather than guessing the fix. Both mechanisms run in a single Curator call, requiring no extra LLM invocation.
3. Empirical evaluation across two optimizer models and four benchmark families: WebArena (GitLab, Shopping, Shopping Admin, Reddit via GoBrowse), ALFWorld, AppWorld, and τ-bench, using Gemini 3.1 Pro Preview and Claude Opus 5. GEPA-Curator matches or exceeds mean performance of both baselines, with gains up to 20.7% over GEPA and 21.2% over ACE.
4. Extension to harness-code optimization and a cost-reduced variant: Meta-Harness-Curator applies the same curation idea to Meta-Harness (which optimizes agent harness code), and an "Efficient GEPA-Curator" adds skip-interval decisions, compacted task histories, and successful-trajectory retention.

Main Findings

• WebArena gains with Gemini 3.1 Pro: Aggregate held-out test success rose from a seed prompt of 72.4 ± 2.4 to 78.7 ± 0.6 for GEPA and 82.1 ± 1.7 for GEPA-Curator. GEPA improved on the seed by 6.3%, GEPA-Curator by 9.7%, giving an aggregate difference of 3.4% and per-website gains of 1.6–7.4%. The paper reports this as 1.54× GEPA's optimization gain.

• WebArena gains with Claude Opus 5: Seed 73.1 ± 1.8, GEPA 79.5 ± 1.2, GEPA-Curator 84.5 ± 1.0. GEPA improved 6.4% over the seed, GEPA-Curator 11.4%, an aggregate difference of 5.0% with per-website gains of 0.6%–7.4%, reported as 1.78× GEPA's gain.

• Per-website WebArena numbers (Gemini / Opus): GitLab 84.6 / 83.1, Shopping 85.8 / 86.5, Shopping Admin 73.1 / 80.3, Reddit 85.0 / 88.2 for GEPA-Curator, versus GEPA's 82.1 / 82.5, 83.8 / 80.5, 71.5 / 72.9, and 77.6 / 81.9.

• Other agentic benchmarks with Gemini 3.1 Pro: GEPA-Curator reached ALFWorld 91.8 ± 1.4, AppWorld Test-N 58.7 ± 1.9, AppWorld Test-C 46.2 ± 2.1, and τ-bench 84.0 ± 4.8, versus GEPA's 82.4 / 38.0 / 27.8 / 75.4 and ACE's 90.1 / 55.7 / 34.5 / 80.6. Average gains were 14.3% over GEPA and 5.0% over ACE.

• Other agentic benchmarks with Claude Opus 5: GEPA-Curator reached 98.4 on ALFWorld (+2.6% over GEPA), 68.8 on AppWorld Test-N (+15.9%), 46.4 on AppWorld Test-C (+13.9%), and tied GEPA at 82.9 on τ-bench. Averaged across the four test sets, gains were 8.1% over GEPA and 12.6% over ACE.

• The two mechanisms are complementary: On a 200-task GitLab ablation (520 training tasks, 200 validation tasks), the seed scored 78.2 ± 2.5, GEPA 82.8 ± 1.6, GEPA without Pareto selection 82.5 ± 1.9, task curation only 84.2 ± 2.9, prior success retrieval only 84.7 ± 3.1, and the full method 88.6 ± 2.0. Each component alone beat GEPA (+1.4% and +1.9%), but combining them was substantially stronger.

• Noisy tasks contaminate optimization: In motivating experiments on GitLab tasks classified as Easy, Optimizable, Impossible, and Flaky, GEPA reached near-perfect accuracy when Optimizable tasks were mixed with Easy tasks, but peaked at only 12/16 when Optimizable tasks were mixed with Impossible or Flaky tasks.

• Binary success/failure history is insufficient: On a mixed dataset of 48 Easy, 54 Optimizable, 45 Impossible, and 21 Flaky tasks, a numerical curriculum using only historical success/failure scored 62.5%, below GEPA's 66.2%. Task curation alone reached 67.4% (4.9% above the numerical curriculum, 1.2% above GEPA), and full GEPA-Curator reached 70.2% (2.8% above curation alone, 4.0% above GEPA). The authors attribute this to execution traces being needed to distinguish evaluator/environment noise from actionable prompt failures.

• Curator diagnosis behavior: Across 1,581 GoBrowse training tasks, the Curator diagnosed 75 tasks (4.7%) as impossible or flaky at least once. Eight tasks received both diagnoses across iterations, leaving 57 unique flaky-only tasks, while the flaky-cause breakdown includes 65 flaky diagnoses. Most impossible-task diagnoses were attributed to harness or environment limitations such as an unavailable required tool; ambiguous or faulty task descriptions were the most common cause of flaky diagnoses.

• Curation transfers to harness-code optimization: Meta-Harness-Curator improved Math-RAG overall test accuracy from 45.33% to 46.33% and mean reward from 45.69% to 47.26%, with ArXivMath January accuracy rising from 26.09% to 34.78% but ProofBench accuracy falling from 35.56% to 33.89%. On text classification, overall accuracy rose from 48.06% to 51.04%, with S2D 84.58 → 88.13 and Law 39.17 → 45.00, while USPTO declined slightly from 20.42 to 20.00.

• A cheaper variant holds up: Efficient GEPA-Curator produced GitLab 83.8, Shopping 85.1, Shopping Admin 73.3, Reddit 86.8, ALFWorld 90.0, AppWorld Test-N 67.1, AppWorld Test-C 56.3, and τ-bench 84.0, comparable to the full method while sometimes exceeding it on AppWorld and Reddit.

• Cost: Estimated optimization costs for one run per benchmark (excluding final test evaluations and WebArena hosting) were $594.05 for GEPA versus $776.88 for Efficient GEPA-Curator on the four WebArena websites (78.7% vs 82.3% overall accuracy). On the non-WebArena benchmarks, GEPA cost $132.65 (55.9% accuracy), ACE $768.69 (65.2%), and Efficient GEPA-Curator $173.98 (74.4%). The paper reports this as a 1.31× cost increase over GEPA for +3.6% on WebArena and +18.5% on non-WebArena, and 4.42× cheaper than ACE with +9.2% higher average accuracy on the non-WebArena benchmarks. The paper notes the ratio would shrink when optimizing a larger actor model.

Methodology in Plain English

The authors treat any prompt optimizer as a black box with a simple interface: give it a prompt and a small batch of tasks, and it runs those tasks with an agent, then returns an updated prompt plus the trajectories and scores. Ordinary optimizers shuffle and walk through the training set, so every task's failure — including failures caused by broken tools or an unreliable judge — can influence the rewrite.

GEPA-Curator wraps this loop from the outside and keeps a per-task memory of every past rollout and its evaluation outcome.

• Before choosing a batch, the Curator reads each task's history. Tasks with little history are prioritized so enough evidence accumulates. Once a task has a clear record, the Curator may skip it for a while — for example if it always succeeds (nothing to learn), always fails for reasons outside the prompt (broken environment or missing tool), or gets contradictory judgements on similar outcomes. Skipping is temporary and reassessed each training epoch, because a prompt update for one task can break another, and previously impossible tasks can become solvable.
• Before reflecting, the optimizer is given one past successful trajectory for the same task, chosen for having consistent evaluation evidence, so the Reflector can compare the failure to a known-good run instead of guessing. If no trustworthy success exists, the Curator abstains rather than supplying a misleading positive example.

Both decisions are produced in a single Curator LLM call and passed to the optimizer as extra metadata, leaving the optimizer itself unmodified.

Experimentally, the authors build on a simplified greedy variant of GEPA without Pareto-based candidate selection, maintaining a single lineage where each mutation is the next checkpoint. They use gpt-oss-120b as the actor, Gemini 3.1 Pro Preview and Claude Opus 5 as the optimizer-side models (Reflector and Curator), minibatch size 16, an actor context limit of 131,072 tokens, manually written seed prompts, validation every 16 iterations (every 4 for τ-bench), and averages over 5 repeated runs. Budgets were 5,000 rollouts per website/method on GoBrowse, 2,000 on ALFWorld, 1,500 on AppWorld, and 800 on τ-bench, chosen so all methods reach a saturated success rate.

Why This Matters

Impact on research. The paper reframes self-improvement as a data selection problem rather than only a reflection quality problem. Existing safeguards such as held-out validation gating act after a bad mutation has been generated, and can themselves misfire on noisy aggregate scores. Showing that a simple outer curator improves results for two different optimizer models, four benchmark families, and even harness-code optimization suggests the field should treat trajectory curation as a first-class component of any reflective optimization loop.

Real-world applications:

• Web and computer-use agents (customer support, form filling, e-commerce checkout) that must keep improving from live traffic where broken pages, flaky buttons, and ambiguous requests are routine.
• Enterprise workflow agents on platforms like AppWorld, where a missing tool permission is an environment problem, not a prompt problem, and should not trigger prompt rewrites.
• Embodied or simulated task agents such as ALFWorld, where a small validation set makes each training example expensive and noisy examples are costly.
• Agent-harness engineering, where the same curation logic can guide a coding agent that edits orchestration code rather than prompts.

Industry relevance. The paper includes an explicit cost analysis in USD using published Together AI and Gemini API rates, showing that curation overhead is modest relative to accuracy gains and far cheaper than the ACE baseline on non-WebArena benchmarks. For teams running continuous prompt optimization in production, that trade-off — a 1.31× cost increase for measurable accuracy gains — is the practical question.

Future Directions

• Learned curation policies: The Curator is currently an LLM reasoning over task history with a hand-written prompt (given in the appendix). Whether a trained or distilled curation policy could match it at lower cost is open.
• Curating beyond prompts and harness code: The paper extends curation to Meta-Harness, but how far the idea generalizes — to memory construction, tool sets, or multi-agent role assignment — is not established.
• Scaling behavior: The authors hypothesize that the division of labor between curator and optimizer becomes more useful as the pool of training tasks and execution histories grows, but this is stated as a hypothesis rather than measured at scale.
• Cost-accuracy trade-off under stronger actors: The paper notes that its cost ratio would shrink significantly when optimizing a larger model than gpt-oss-120b, but does not report that regime.

Target Audience

Researchers and practitioners working on LLM agents, agentic benchmarks, and automatic prompt or context optimization. It is most useful to readers who already run, or plan to run, a reflective optimization loop on real agentic data and have observed unstable or harmful prompt updates. Readers new to the area can follow the core argument — that not every failure is worth learning from — without prior background in the specific optimizers.

Authors’ abstract

Large Language Models (LLMs) are increasingly used in multi-turn agentic workloads that require sustained interaction with complex environments. Existing reflective prompt optimizers such as GEPA learn from failed rollouts, but agentic failures can also come from prompt weaknesses, harness limitations, stochastic execution, or noisy judges. We argue that such failures may not provide useful learning signal, and may even be harmful; and thus it is essential to curate which tasks are provided to the prompt optimizer for better performance. We introduce GEPA-Curator, a framework that augments the prompt optimizer to select which agent experiences should shape prompt optimization. In particular, GEPA-Curator consists of two components: task curation, which filters out tasks that are likely to provide noisy or unreliable learning signals, and prior success retrieval, which selectively retrieves past successful trajectories for a task and provides them as additional context to the prompt optimizer. We evaluate GEPA-Curator on GoBrowse, ALFWorld, AppWorld, and $τ$-bench, using both Gemini 3.1 Pro and Claude Opus 5 as optimizer models. GEPA-Curator matches or exceeds the mean performance of both baselines, with gains of up to 20.7% over GEPA and 21.2% over ACE.