AgentLTV: An Agent-Based Unified Search-and-Evolution Framework for Automated Lifetime Value Prediction

AI summary

Overview

Research area: Machine learning for user Lifetime Value (LTV) prediction, combining large language model (LLM) driven multi-agent code generation with Monte Carlo Tree Search (MCTS) and Evolutionary Algorithms (EA).

Technical level: Advanced. The paper assumes familiarity with LTV modeling, tree search, evolutionary search, and LLM agent orchestration.

One-sentence scope: The paper introduces AgentLTV, an LLM-agent framework that automatically searches for and evolves executable LTV prediction pipelines, and reports offline, ablation, and online bucket-level results on a large proprietary mobile-game dataset plus a public benchmark.

What This Paper Is About

LTV prediction — estimating the net monetary value a user will generate — is important in advertising, recommender systems, and e-commerce, but in practice each business scenario has different data patterns, so practitioners end up hand-building complex, scenario-specific pipelines and iterating over feature processing, objective design, and tuning. That process is expensive and transfers poorly to new scenarios. AgentLTV's goal is to automate this work: instead of a human choosing a model family, LLM-driven agents generate, execute, repair, and evolve complete LTV pipeline programs, searching broadly with MCTS and then refining with an island-based evolutionary algorithm.

Key Contributions

1. A unified search-and-evolution framework for LTV modeling. AgentLTV searches over executable LTV pipeline programs without assuming a fixed model family, combining MCTS for broad exploration with an EA stage for refinement, so it can adapt to different data patterns. The authors state it has been successfully deployed online.

2. Execution-grounded multi-agent auto-coding for end-to-end automation. The paper claims to be the first to apply an LLM-driven multi-agent system to LTV modeling. Two decision agents (MCTS Agent and EA Agent) orchestrate the process, while auxiliary agents (Code Generator, Code Fixer, Adviser Agent, Expert Optimizer) generate code, run and repair pipelines, and propose grounded edits.

3. Handling of negative LTV. The paper highlights that most existing methods do not explicitly handle negative LTV because they use the zero-inflated log-normal (ZILN) loss, whereas LTV can be negative when the platform spends on a user with little or no return. AgentLTV's discovered pipeline includes negative-aware output design and calibration.

4. Practitioner-oriented guidance. The authors recommend using MCTS when data patterns or constraints change and using EA for stable refinement under a fixed budget, and validating both ranking and calibration through bucket-level diagnostics before deployment.

Main Findings

• Offline superiority over ten baselines. On the proprietary mobile-game dataset, AgentLTV achieves the best results across all four metrics (Error Rate, Norm GINI, Spearman, RMSE) in both periods. Period 1: ER 0.705 ± 0.162, Norm GINI 0.963 ± 0.008, Spearman 0.832 ± 0.014, RMSE 133.58 ± 28.23. Period 2: ER 0.875 ± 0.138, Norm GINI 0.959 ± 0.005, Spearman 0.826 ± 0.017, RMSE 145.61 ± 16.24.

• Magnitude of improvement over the strongest baseline. Compared with the strongest baseline, AgentLTV reduces ER by 41.26% on average, improves Norm GINI by 15.44%, improves Spearman by 40.97%, and reduces RMSE by 47.82%.

• The best program emerged at depth 3 in the MCTS tree. The search shows a wide-but-shallow pattern: because the first-layer nodes are domain-informed initial programs and branching is large under a limited budget, MCTS prioritizes breadth to identify promising model families. The authors interpret this as evidence that most LTV gains come from a few high-impact structural decisions (model family, key design choices), while deeper expansions yield finer-grained, diminishing, noisier returns.

• Both stages matter, but differently. Removing the EA stage changes performance noticeably (ER 0.670, Norm GINI 0.958, Spearman 0.823, RMSE 185.95 versus full AgentLTV at ER 0.594, Norm GINI 0.962, Spearman 0.817, RMSE 141.09). Removing the MCTS stage causes a much larger drop in ranking-related metrics (ER 1.030, Norm GINI 0.778, Spearman 0.633, RMSE 273.42), indicating broad exploration is critical for finding scenario-specific pipelines in a large search space.

• Search scale. The MCTS stage explores 133 nodes and stops after 100 feasible, executable node programs; the EA stage uses 4 islands and stops after generating 100 additional feasible programs, evaluating 116 additional candidates. Table 6 in Appendix C lists the top-20 candidates from the EA stages.

• Online bucket-level gains, especially at the extremes. Analyzing real traffic logs from July 2025, AgentLTV shows the strongest ranking quality (same-interval ratio) in the high-value bucket and the negative-LTV bucket, and lower MAE in most buckets, indicating better value calibration.

• The best discovered model's structure. On this dataset, the winning pipeline includes temporal convolution and attention branches for dynamic behaviors, together with negative-aware output design and calibration.

• Public benchmark. The abstract states that experiments on a large-scale proprietary dataset and a public benchmark show AgentLTV consistently discovers strong models across ranking and error metrics. The specific public-benchmark result values are not reported in the available paper content, which is truncated at Table 4 (basic statistics of the public dataset).

Methodology in Plain English

AgentLTV treats each candidate solution not as a set of hyperparameters but as an executable pipeline program — a runnable piece of code that loads data, trains, evaluates, and logs. The framework has three modules:

1. Configuration module. Instantiates a team of LLM-driven agents with role-specific prompts, using multiple LLM APIs (claude-sonnet-4.5, gpt-5.2, and gemini-3-pro). Two are decision agents (MCTS Agent and EA Agent); the rest are auxiliary: a Code Generator (writes or edits pipeline code), a Code Fixer (runs training and evaluation, handles failures, repairs code iteratively), an Adviser Agent (analyzes code structure and execution results and proposes optimizations), and an Expert Optimizer (injects LTV domain knowledge and converts suggestions into interface-consistent prompts). All agents share a unified pipeline interface so candidate programs can be compared fairly.

2. MCTS module (broad exploration). Starting from K diverse root programs generated from dataset descriptions and LTV domain knowledge, the MCTS Agent grows a search tree under a fixed budget. Node selection uses the PUCT criterion combined with a Pareto-aware multi-metric reward: R(c) = Σ wᵢ · norm(oᵢ(c)) + β · I_Pareto(c) + γ · d_crowd(c), where the first term aggregates normalized evaluation metrics, the second flags a node on the Pareto frontier, and the third rewards diversity via crowding distance. Each expansion has the Adviser Agent propose edits, the Expert Optimizer score and reweight them into a prior P(c), the Code Generator write the child program, and the Code Fixer execute and repair it. Values are backpropagated by incremental averaging, and the best node by Q(c) is returned as c_MCTS.

3. EA module (refinement). The best MCTS program seeds K islands as mutated variants. Each island evolves for T generations: evaluate fitness, discard non-executable programs, select elites by ratio ρ, then generate offspring through prompt-guided crossover and mutation. Mutations cover feature augmentation, architecture refinement, hyperparameter tuning, and loss substitution. Every τ generations, high-fitness programs migrate between islands to share building blocks while preserving diversity. The globally best program across all islands is the final output.

Experimental setup: A large-scale proprietary dataset from a popular mobile game, predicting cumulative revenue over the next 60 days from the first 7 days after installation. Two quarters, roughly 800,000 users, 132 features (user attributes such as age, gender, region, city; advertising attributes such as channel, ad type, install info; game behaviors such as activity signals, competition scores/ranks, and behavior across match modes). Period 1 has 426,412 samples (188,047 positive, 146,686 negative, average LTV 23.810); Period 2 has 382,081 samples (171,554 positive, 132,201 negative, average LTV 25.323). Each period is split 8:1:1 into train/validation/test. Baselines: Wide&Deep, DeepFM, ZILN, DCN, GateNet, Kuaishou, TSUR, OptDist, USE-LTV, and Hi-LTV. Experiments ran on a Linux server with two NVIDIA Tesla A100 GPUs and 128GB RAM. The public benchmark is the Kaggle Acquire Valued Shoppers Challenge, predicting total purchase value within 12 months after the first transaction, following the ZILN protocol.

Why This Matters

Impact on research. AgentLTV is positioned as a shift away from hand-designed, scenario-specific LTV architectures toward automated search over whole pipeline programs. It also explicitly targets negative LTV, which the authors argue violates assumptions baked into the widely used ZILN loss, and it combines breadth-oriented search (MCTS) with depth-oriented evolution (EA) in a single loop — a combination the paper argues existing LLM code-generation paradigms have not achieved for LTV because breadth methods replicate existing pipelines without producing accurate models, while evolutionary methods assume a stable initial program that is hard to obtain in realistic LTV settings.

Real-world applications:

• Online display advertising bidding, where per-user value estimates drive bid optimization and where unclicked impressions create negative-value users.
• Short-video and content platforms, where cash-out incentives can exceed user contributions, producing negative LTV.
• Mobile games and retail, where the goal is to identify high-value "whale" users and allocate traffic and marketing spend accordingly.
• E-commerce recommender systems and long-term business planning, where LTV informs targeting and profit optimization.

Industry relevance. The framework is tested on roughly 800,000 users of a real mobile game, validated on real traffic logs from July 2025 at the bucket level, and the authors state AgentLTV has been successfully deployed online. The value proposition is reduced manual engineering: the framework searches over feature processing, model structure, and training recipes without a human hand-designing each pipeline, and produces a single deployable program per scenario.

Future Directions

• Determining when EA pays off relative to its cost. Ablations show EA produces a smaller drop when removed than MCTS, and the paper's own guidance is to use EA only for stable refinement under a fixed budget — the boundary between the two regimes is left open.
• Explaining and controlling the wide-but-shallow search pattern. Since the best program appeared at depth 3, an open question is whether the observed shallowness is a property of LTV modeling generally or of the specific budget, rewards, and prompt priors used here.
• Extending the public-benchmark evaluation. The paper states a public-benchmark result exists and that it follows the ZILN protocol on the Acquire Valued Shoppers Challenge dataset, but the detailed results are not present in the available content, so broader generalization across further scenarios remains to be examined.
• Quantifying the online economic value more fully. The online analysis reports same-interval ratio and per-bucket MAE; connecting these bucket-level diagnostics to revenue or profit outcomes is a natural extension.

Target Audience

This paper is most useful to applied machine learning engineers and data scientists who build and maintain LTV or user-value prediction systems in advertising, gaming, e-commerce, and recommender settings; to industrial researchers evaluating LLM-agent-based automated machine learning pipelines; and to academic researchers working on automated code generation with execution feedback, MCTS-guided search, or evolutionary program optimization who want a concrete, production-scale application case.

Authors’ abstract

Lifetime Value (LTV) prediction is critical in advertising, recommender systems, and e-commerce. In practice, LTV data patterns vary across decision scenarios. As a result, practitioners often build complex, scenario-specific pipelines and iterate over feature processing, objective design, and tuning. This process is expensive and hard to transfer. We propose AgentLTV, an agent-based unified search-and-evolution framework for automated LTV modeling. AgentLTV treats each candidate solution as an {executable pipeline program}. LLM-driven agents generate code, run and repair pipelines, and analyze execution feedback. Two decision agents coordinate a two-stage search. The Monte Carlo Tree Search (MCTS) stage explores a broad space of modeling choices under a fixed budget, guided by the Polynomial Upper Confidence bounds for Trees criterion and a Pareto-aware multi-metric value function. The Evolutionary Algorithm (EA) stage refines the best MCTS program via island-based evolution with crossover, mutation, and migration. Experiments on a large-scale proprietary dataset and a public benchmark show that AgentLTV consistently discovers strong models across ranking and error metrics. Online bucket-level analysis further indicates improved ranking consistency and value calibration, especially for high-value and negative-LTV segments. We summarize practitioner-oriented takeaways: use MCTS for rapid adaptation to new data patterns, use EA for stable refinement, and validate deployment readiness with bucket-level ranking and calibration diagnostics. The proposed AgentLTV has been successfully deployed online.