LCM-Promo: An LLM-Based Uplift Model for Promotion Allocation

AI summary

Overview

Research area: Applied machine learning for e-commerce — specifically uplift modelling and promotion targeting using a domain-specialized, fine-tuned large language model (a "Large Commerce Model").

Technical level: Intermediate. The paper assumes familiarity with uplift modelling, cumulative response curves, A/B testing, and basic LLM fine-tuning and decoding, but the core ideas are explained clearly enough for a reader with general ML background.

Scope: The paper describes LCM-Promo, a 4B-parameter fine-tuned LLM that predicts user conversion thresholds from commerce data, and reports both a one-month production A/B test at iFood and an offline uplift comparison against an Uplift Random Forest.

What This Paper Is About

Promotions in food delivery are ephemeral and context-dependent — the right discount depends on the day, the delivery shift, the product category, and the user's history — which makes traditional per-segment forecasting policies coarse and slow to adapt. The paper frames promotion targeting as uplift modelling (estimating how much more likely a user is to convert with a given promotion than with none) and asks whether a fine-tuned LLM can produce these response curves directly from serialized user, session, and discount information. The goal is a single production model that supports finer-grained, budget-constrained allocation of promotions than the incumbent Segment-EMA system.

Key Contributions

1. A task-specific Large Commerce Model for promotion allocation. LCM-Promo fine-tunes a 4B-parameter dense model on transactional commerce data, conditioning on a system prompt, session context (such as day and delivery shift), an LCM-generated user representation, and a list of discount options mapped to alphabetic identifiers.

2. A decoding scheme that yields monotonic response curves by construction. The model predicts a single token over discount identifiers, which the authors renormalize into a probability mass function over a latent conversion threshold. Summing that PMF produces a cumulative conversion-probability curve that is non-decreasing by construction and can therefore be used directly for allocation and offline uplift evaluation.

3. A production A/B test at iFood. Over one month on churn reactivation users across Brazil, with a 90% control group on Segment-EMA and a 10% treatment group on LCM-Promo (identical budget constraint and identical linear-optimization allocator), LCM-Promo delivered a 1% relative conversion lift and 6% lower promotion cost. The system was subsequently rolled out to all churn users.

4. An offline benchmark against an Uplift Random Forest on RCT data, plus an ablation of a "waterfall" label augmentation, showing the LLM approach is competitive on AUUC and Qini but below the random forest on Uplift@30, and that the waterfall variant performs worse.

Main Findings

• Online conversion lift: In the one-month production A/B test on churn reactivation, LCM-Promo increased conversion by 1% relative to the Segment-EMA baseline.

• Online cost reduction: The same test reduced promotion cost by 6% relative to Segment-EMA, meaning better outcomes at lower spend.

• Massive expansion of the allocation space: Because delivery shift and other session contexts are prompt inputs rather than separate forecasting cells, LCM-Promo expanded allocation from about 50 segments to over 1,000 clusters, and from 10 wallet configurations to over one million promotion combinations.

• Offline competitiveness on AUUC and Qini: On randomized controlled trial data with a fixed promotion, LCM-Promo (post-processing) reached AUUC 0.0608 ± 0.0032 and Qini 0.0490 ± 0.0028, versus 0.0578 ± 0.0064 and 0.0466 ± 0.0053 for the Uplift Random Forest.

• Offline shortfall on Uplift@30: The Uplift Random Forest scored higher on Uplift@30 at 0.1708 ± 0.0064, compared with 0.1626 ± 0.0035 for LCM-Promo (post-processing).

• Waterfall augmentation hurts: The waterfall label variant scored 0.0278 ± 0.0036 AUUC, 0.0226 ± 0.0029 Qini, and 0.1208 ± 0.0026 Uplift@30 — worse across all three metrics — which the authors attribute to its multi-label soft targets conflicting with the LLM's next-token pretraining objective.

• Feasible serving cost: Inference runs twice daily (lunch and dinner) on a self-hosted vLLM deployment via BESH, the authors' open-source batch endpoint for self-hosted GPUs, running 1 hour daily on 8 H200s. Feasibility came from clustering users by city, session, and lifecycle and estimating mean response curves by random sampling with bootstrap-determined sample sizes.

Methodology in Plain English

The team treats promotion response as a next-token prediction problem. For each user, they build a prompt containing a system instruction, the session context (day, delivery shift), a structured behavioural summary of the user produced by a separate LCM representation model, and the available discount options — each discount mapped to a letter. The 4B-parameter model is trained with cross-entropy to output the letter corresponding to the discount at which the user actually converted, using a special "no purchase" token for users who did not convert.

At inference, the model's next-token probabilities are restricted to the valid discount identifiers and the no-purchase token, then renormalized. The authors read this distribution as a probability distribution over a user's hidden conversion threshold: the minimum discount at which they would buy, or "never" if no available discount is enough. Adding up the probabilities from the smallest discount up to a given one gives the probability the user converts at that offer — and because it is a running sum, the resulting curve can only go up. That monotonicity makes the curves directly usable for budget-constrained optimization and for standard offline uplift metrics.

They also test a "waterfall" labelling scheme that marks every discount at or above the observed conversion discount as positive, smoothing these into soft targets. In production, users are grouped into clusters and mean response curves are estimated by sampling rather than scoring every user, which is what made running the system twice a day practical. Online, the model was compared against iFood's prior policy, Segment-EMA, which forecasts orders per segment-promotion cell with an exponentially weighted moving average and therefore has to explore each new cell separately, limiting how fine-grained segmentation can be.

Why This Matters

Impact on research: The paper offers a concrete recipe for turning an LLM into an uplift model without a bespoke architecture: threshold-style single-token decoding gives monotonic response curves for free, and appears to outperform a multi-label "waterfall" alternative. It is also a rare industrial report of an LLM uplift model validated in a live randomized test rather than only offline, and it contributes an open-source serving tool (BESH) for self-hosted batch inference.

Real-world applications:

• Churn reactivation in food delivery — the deployment reported here, where the system replaced a segment-level policy for all churn users at iFood.
• Promotion and discount budgeting — monotonic response curves plug directly into budget-constrained allocators, as demonstrated with the same linear-optimization allocator used by the baseline.
• Context-sensitive offers in marketplaces — day, delivery shift, category, and discount mechanic can be serialized into the prompt, so fine-grained contexts do not require separate forecasting cells.
• Other commerce lifecycle stages — the authors frame the method as a template for targeting and merchant operations decisions beyond churn.

Industry relevance: The result is an efficiency story as much as an accuracy story: 1% more conversions at 6% lower promotion cost, with a serving footprint of 1 hour daily on 8 H200s. That combination — better outcomes, lower spend, tractable infrastructure — is what makes the case for replacing cell-based forecasting with a learned response model in production promotion systems.

Future Directions

• Extending beyond one lifecycle stage and one platform. The authors explicitly state that the results are limited to a single lifecycle stage (churn reactivation) and a single platform (iFood), and that extending to other lifecycle stages and commerce platforms is ongoing work.

• Reconciling the offline and online pictures. LCM-Promo beat the Uplift Random Forest on AUUC and Qini but trailed on Uplift@30, and the offline benchmark used a fixed promotion while production spans multiple depths and contexts. A benchmark that better mirrors the multi-depth production setting would clarify when the LLM is genuinely the stronger uplift model.

• Fixing or replacing the waterfall augmentation. The waterfall labelling underperformed across all three offline metrics, which the authors attribute to a mismatch with next-token pretraining. Alternative ways of encoding threshold ordering in the labels — or in the decoding — remain open.

• Understanding where the threshold proxy breaks. The method uses the observed conversion discount as a proxy for the latent conversion threshold, justified by the assumption that users typically convert at most once per month. Behaviour outside that assumption (repeat converters, stacked offers) is not addressed and would need further study.

Target Audience

This paper is most useful to applied ML scientists and engineers working on uplift modelling, causal targeting, or promotional optimization in industry settings, and to teams weighing whether to specialize an LLM for a narrow decision task rather than maintain a hand-engineered forecasting pipeline. It also suits recommender and marketing-analytics practitioners who care about the practical trade-offs of LLM serving — prompt serialization, batch inference economics, and sampling-based approximation. Researchers interested in how LLMs can be repurposed as structured probabilistic predictors, rather than text generators, will find the threshold-decoding construction the most transferable idea. Readers looking for a controlled benchmark study should note the evaluation is deliberately narrow: one lifecycle stage, one platform, and an offline comparison on fixed-promotion RCT data, with no dataset sizes reported for the training logs.

Authors’ abstract

We present LCM-Promo, an LLM-based uplift model for promotion allocation. LCM-Promo is a task-specific Large Commerce Model (LCM): a finetuned LLM trained on transactional commerce data to predict user intent and promotion response. It learns monotonic promotion-response curves from production logs by interpreting single-token outputs as conversion-threshold probabilities, while conditioning on session context such as day and delivery shift. In a production A/B test on churn reactivation, LCM-Promo increased conversion by 1% and reduced promotion cost by 6% relative to the Segment-EMA production baseline.