EvolveTrade: Experience-Driven Policy Refinement for Self-Evolving LLM Trading Agents

AI summary

Overview

Research area: LLM-based autonomous financial trading agents, self-evolving/self-improving LLM agent design, and prompt-as-policy optimization.

Technical level: Intermediate. The paper assumes familiarity with LLM tool-calling agents, prompt optimization, and standard portfolio evaluation metrics (Sharpe Ratio, cumulative return, maximum drawdown).

Scope: This paper proposes and empirically tests a framework in which a tool-using LLM trading agent rewrites its own system-prompt-level tool-use policy from accumulated decision traces and realized portfolio feedback, while the underlying LLM weights remain frozen.

What This Paper Is About

LLM trading agents that retrieve prices, search news, and run code are usually governed by a hand-written system prompt that is fixed before deployment, so the agent cannot learn from how its own trades actually turned out. EvolveTrade addresses this by treating the system prompt as a learnable "text policy" and having a separate Policy Agent rewrite it after each interval of trading, using the agent's own reasoning traces and realized portfolio outcomes. The goal is to let the agent improve how it gathers, verifies, and acts on evidence, rather than fine-tuning the model or changing its tools.

Key Contributions

1. A self-evolving framework for tool-use policy: EvolveTrade treats the trading agent's system prompt as a text-parameterized policy π_t over evidence gathering, tool use, risk control, and portfolio construction, and revises it online while the backbone LLM parameters and tool interfaces stay fixed.

2. Experience-as-feedback formulation: The paper pairs each day's decision trace h_t = (w_t, d_t^dec) — the submitted allocation plus per-asset rationale — with post-trade feedback g_t (daily portfolio return, per-asset returns, resulting allocations and portfolio state) into a single refinement record R_t = (h_t, g_t), explicitly to disambiguate noisy feedback.

3. A batch-wise refinement step: A separate Policy Agent applies f_update(π, B) = LLM(I_update, π, B) at a fixed interval of N ≥ 1 trading days, producing a sequence of batch policies where each later policy has been shaped by the agent's own realized experience.

4. Diagnostic and case-level evidence: Behavioral analyses show which analytic metrics and tools the evolved policy activates (including metrics the fixed-policy baseline never invokes), and a case-level policy-to-return attribution links one policy-induced sizing change to realized return differences.

Main Findings

• Self-evolution often improves risk-adjusted trading performance: Across two backbones (GPT-5-mini and Gemini-2.5-Flash) and six post-cutoff market regimes, EvolveTrade achieves the best Sharpe Ratio and Cumulative Return among LLM-based methods in three model-regime pairs, and the paper states fixed-policy agents remain stronger in two others.

• GPT-5-mini results: EvolveTrade is strongest in January 2025 (SR 5.12, CR 5.10%, MDD 2.39%, WR 57.0%, Vol 0.84) and September 2025 (SR 8.43, CR 6.84%, MDD 1.26%, WR 81.3%, Vol 0.65), achieving the best SR and CR among LLM agents in both periods. In April 2025 it records SR -2.53 and CR -6.60%, below the Static Tool-Calling Agent's -1.54 and -4.71%.

• Gemini-2.5-Flash results: EvolveTrade performs best in November 2025 (SR -1.03, CR -1.07%) and second-best in February 2026 (SR 2.75, CR 2.92%), behind EvolveStrategy (SR 3.92, CR 3.52%). In April 2026 it records SR 4.73 and CR 3.69%, below Static Base Agent's 9.07 and 11.64%.

• It can beat rule-based baselines on return-side metrics: With GPT-5-mini in September 2025, EvolveTrade's CR of 6.84% exceeds the strongest rule-based CR in that period. With Gemini-2.5-Flash in February 2026, EvolveTrade achieves SR 2.75 and CR 2.92%, outperforming all traditional rule-based baselines.

• Static policies lock the agent into a fixed analytic vocabulary: Under a fixed policy with GPT-5-mini, the same five metrics (Sharpe, 20d return, 60d return, annualized volatility, drawdown) appear on 90–100% of days in all three regimes tested (Jan. 2025 sideways, Apr. 2025 drawdown of -9.8% with V-shaped recovery, Sep. 2025 steady uptrend), while RSI, EMA, VaR, and signal normalization never appear at all. News queries reduce to per-asset boilerplate (e.g. "TICKER", "TICKER news") and price-retrieval lookback is fixed at roughly six months on every call.

• EvolveTrade activates regime-relevant metrics the baseline never invokes: Tail-risk metrics (VaR and signal normalization) appear in the April drawdown, and trend-following indicators (SMA, EMA, RSI) appear in the September uptrend, with the baseline invoking none of these in any of the three regimes.

• It shifts tool-use from text gathering toward executable analysis: The price-retrieval tool stays close to one call per day in both settings, and news search does not increase overall, but code execution rises from about 1.0 to 2.6–4.5 calls per day across regimes under EvolveTrade.

• Case-level attribution in the January 2025 NVDA drawdown: On January 24, the Static Tool-Calling Agent held 10.7% NVDA while EvolveTrade held 2.9% under a refined sizing policy prioritizing longer-horizon risk-adjusted evidence. NVDA fell 17.0% the next trading day; EvolveTrade returned -0.03% versus -1.11% for the Static Tool-Calling Agent. The abstract describes this as a +1.33 percentage-point relative daily return difference.

• Effectiveness persists over a longer horizon: Over 50 trading days with GPT-5-mini (2025-09-02 to 2025-11-10), EvolveTrade attains SR 4.00 and CR 10.56% (MDD 2.96%, WR 67.3%, Vol 0.81), best among all listed methods, versus Static TC Agent's SR 2.94 and CR 8.88%. Policy length grows but contracts periodically across the three runs, consistent with consolidating rather than indefinitely accumulating instructions.

• Moderate update intervals perform best: Averaged over January, April, and September 2025, the 5-day interval yields the highest average SR (3.67) and CR (1.78%), while the 1-day interval yields the highest WR (61.5) and a lowest Vol (1.13) tied with 3 days. Daily updates produce the lowest average SR (1.92) and CR (0.16%), suggesting overly frequent revision overfits short-horizon feedback.

• Gains survive 10 bps transaction costs: In the turnover-and-cost table, EvolveTrade retains the highest SR and CR among LLM-based methods in the same three settings where it leads in the main results, with turnover of 1.05 (Jan. 2025), 1.68 (Sep. 2025), and 1.93 (Nov. 2025).

• Run-to-run reliability: Across three runs, EvolveTrade's standard deviations are generally comparable to those of other policy-evolving agents with no systematic increase; e.g. GPT-5-mini January 2025 SR 5.12 ± 0.51 and CR 5.10 ± 0.72, September 2025 SR 8.43 ± 1.68 and CR 6.84 ± 0.70.

Methodology in Plain English

The researchers build a daily-close trading simulation based on the LiveTradeBench framework, in which an agent sees a portfolio state, submits target portfolio weights, and the portfolio is rebalanced at the closing price. The investment universe is 15 major US blue-chip stocks (e.g., AAPL, MSFT, NVDA, JPM) plus cash, with a long-only allocation simplex unless otherwise specified.

The agent has three tools: price retrieval, news search, and a Python code interpreter — all retrieval tools enforcing the same temporal cutoff to prevent look-ahead leakage. Market data is deliberately not pre-injected into the prompt; the agent must retrieve it itself. Each day the agent produces an allocation w_t plus a per-asset rationale d_t^dec.

Every N trading days (N = 5 in the main experiments), a separate Policy Agent — an LLM call with a fixed update instruction — reads the batch of refinement records (decisions, rationales, and realized returns) and rewrites the full policy text. The update instruction asks it to identify which aspects of the current prompt contributed to good or poor performance and to link concrete observations to specific lines of the prompt, favoring grounded edits over generic rewrites. A refinement step can change tool selection, query formulation, signal interpretation, evidence verification, risk control, or output structure, but never the LLM weights or tool interfaces.

Evaluation covers one-month online policy-updating windows (following DeepFund's live evaluation length). GPT-5-mini is tested on January, April, and September 2025 (sideways, drawdown-recovery, uptrend) and Gemini-2.5-Flash on November 2025, February 2026, and April 2026 (bearish, sideways, bullish) to respect its later data cutoff. Baselines include rule-based strategies (SPY, Buy and Hold, MACD, KDJ&RSI, ZMR, SMA) and LLM configurations that isolate each factor: Static Base Agent (no tools), Static Tool-Calling Agent (tools, fixed policy), EvolveBase (evolution without tools), EvolveStrategy (strategy-level prompt evolution adapted from ATLAS, with tool protocols fixed), and EvolveTrade (full policy evolution focused on tool-calling protocols). Each LLM configuration is run three times and averaged.

Why This Matters

Impact on research: The paper reframes the bottleneck for LLM trading agents away from model capability, context richness, or tool access, and toward the reusable procedure governing how tools are used. It also extends the self-evolving LLM agent literature — prompt optimization, reflective memory, learned workflows — to a financial setting where feedback is delayed, noisy, and non-stationary, and it introduces a concrete per-day record design (trace plus realized feedback) to handle that ambiguity. By deliberately optimizing only the policy text, it offers a route to adapting deployed agents that does not require retraining or modifying tools.

Real-world applications (implications drawn from the framework; the paper evaluates only simulated daily-close trading):

• Quantitative portfolio management, where a deployed agent's analytic emphasis could shift with market regime (tail-risk metrics in drawdowns, trend indicators in uptrends).
• Risk management and drawdown control, illustrated by the January 2025 NVDA case where a refined sizing policy reduced exposure from 10.7% to 2.9% before a 17.0% single-day drop.
• Post-deployment adaptation of financial agents without retraining, since only policy text changes while the backbone LLM stays frozen.
• Broader tool-using agent systems (research assistants, data analysis pipelines) where the question of when, why, and how to invoke tools matters as much as which tools exist.

Industry relevance: The results speak directly to firms deploying LLM agents in non-stationary environments, and the paper reports operating characteristics practitioners care about — update frequency trade-offs, turnover, and performance after 10 bps transaction costs. It is also candid about what is not captured: slippage, market impact, and liquidity constraints are acknowledged as outside the scope of the proportional-cost estimates.

Future Directions

• Dynamically scheduled policy updates: The paper keeps the update interval fixed within each run and identifies updates scheduled in response to market changes as a promising direction.

• More realistic execution modeling: The current transaction-cost analysis uses proportional 10 bps costs and does not capture slippage, market impact, or liquidity constraints; the authors suggest incorporating these through more realistic execution models.

• Generalizing beyond the tested settings: Results cover two backbones (GPT-5-mini, Gemini-2.5-Flash), a 15-stock US blue-chip universe plus cash, one-month evaluation windows, and a 50-day longer-horizon check, leaving broader universes, asset classes, and longer horizons open.

• Understanding when evolution helps and when it does not: EvolveTrade leads in three of six model-regime pairs while fixed-policy or strategy-level evolution baselines lead in others, so identifying which regime and backbone conditions favor tool-policy evolution versus simpler static or strategy-level policies remains an open question.

Target Audience

Researchers working on LLM agents, self-improving/self-evolving agent systems, and prompt or workflow optimization; quantitative finance and algorithmic-trading researchers interested in LLM-based decision-making; and practitioners building or deploying tool-using LLM agents in markets or other non-stationary, feedback-delayed domains. Readers without background in LLM agent architectures or portfolio evaluation metrics will find the experimental tables dense, but the framework, motivation, and behavioral analyses are accessible at an intermediate level.

Authors’ abstract

Large language model (LLM) trading agents can combine market data, news, and executable analysis, but their behavior is often controlled by static hand-written tool-use policies that are fixed before deployment. This limits their ability to adapt how they gather evidence, invoke tools, verify signals, and manage risk under changing market regimes. We introduce EvolveTrade, a self-evolving framework that treats the system prompt of a tool-using trading agent as a text-parameterized policy. After each update interval, a Policy Agent revises this policy using accumulated decision traces and realized portfolio feedback, while keeping the backbone LLM fixed. The updated policy is then used for the next batch of trading decisions, enabling the agent to refine its information-acquisition and portfolio-construction procedure over time. Experiments across multiple market regimes and two LLM backbones show that EvolveTrade often improves Sharpe Ratio and Cumulative Return over fixed-policy LLM baselines, achieving the improved SR and CR in most evaluated settings. Behavioral analyses further show that self-evolved policies increase code-mediated analysis and activate regime-relevant computations; case-level policy-to-return attributions trace how policy-induced allocation changes contribute to realized return differences. These results suggest that adapting the reusable procedure governing tool use is a key direction for building more robust LLM trading agents.