GLOBE: Evolving Structured Agentic Memory for E-commerce Product Optimization in Generative Search

AI summary

Overview

Research area: Natural Language Processing — specifically LLM-based agents, generative engine optimization (GEO), and self-evolving agent memory for e-commerce search.

Technical level: Intermediate. The paper is readable without deep mathematics, but it assumes familiarity with LLM agents, prompting, rank-based evaluation, and embedding similarity.

Scope: The paper proposes GLOBE (GeneraLizable Optimization Behavior Evolution), a framework that stores product-listing optimization knowledge as an evolving handbook of explicit, field-scoped editing behaviors learned from ranking feedback, and evaluates it on the E-GEO benchmark spanning six product categories.

What This Paper Is About

Generative search engines now compare and rank products by reasoning over listing text, so sellers need agents that can rewrite listings effectively — and keep improving by remembering what worked. Existing methods rewrite an entire listing with one monolithic prompt, which both ignores the field structure of product data (title, features, description) and leaves successful tactics buried in prompts or one-off outputs instead of reusable knowledge. GLOBE addresses this by decomposing optimization into named, reusable behaviors that are each restricted to specific fields and refined over repeated batches using rank-change feedback.

Key Contributions

1. A structured agentic memory for product optimization. GLOBE represents optimization knowledge as a handbook of field-scoped behaviors, where each behavior specifies an editing instruction, the listing fields it may touch (a subset of title, features, description), conflicting behaviors, and feedback metadata — separating what to change from where the change is allowed.

2. A feedback-driven evolution loop. Episode-level rank outcomes are converted into persistent editing knowledge through three roles: a Reflector that proposes structured Add/Modify deltas, a Moderator that removes semantically and functionally redundant proposals, and a Curator that deterministically commits accepted changes and recomputes which behaviors remain active.

3. A safety-oriented active-status rule. Behaviors are tracked with helpful/harmful/neutral counts and a score of helpful minus harmful; behaviors with negative scores after a probation period are excluded from the active pool but retained for traceability, giving the framework an explicit mechanism against consistently harmful strategies.

4. Evidence of effectiveness, stability, and transfer. The paper reports gains over monolithic prompt optimization and a static handbook, shows moderated evolution reduces downside risk and category variance while producing a smaller handbook, and demonstrates that learned behaviors transfer to unseen product categories and to different ranking models.

Main Findings

• Structured behaviors beat one big prompt. On the GPT-5.4 ranker, the non-evolving Static HB (9 generic rules) already reached MRG 1.343 and SR 58.6%, versus MRG 0.351 and SR 44.1% for Prompt Opt., while also lowering DSR from 3.961 to 2.628. The authors attribute this to explicit operations plus field scoping.

• Feedback-driven evolution adds further gains, but unmoderated growth is risky. GLOBE w/o Mod. reached MRG 1.519 and SR 60.6%, but grew to 51 behaviors and had higher category variance (CV 0.054) than the static handbook (0.042).

• Moderation acts as handbook-level regularization. Full GLOBE retained fewer behaviors (47) than the unmoderated variant (51) yet achieved the best results on the GPT-5.4 ranker: MRG 1.611, SR 61.6%, DSR 2.172, CV 0.034.

• Gains hold across different ranking models. With the editor and handbook frozen, GLOBE still led on Qwen3.5-122B-A10B (MRG 1.522, SR 54.1%, DSR 0.105) and Gemini-3.5-flash (MRG 1.078, SR 48.9%, DSR 0.246). Prompt Opt. turned negative on both held-out rankers (−0.031 and −0.051 MRG), suggesting it overfits the training engine.

• The moderation gap widens on unseen rankers. GLOBE w/o Mod. lost its clear advantage over the static baseline on held-out rankers, which the authors read as evidence that unfiltered accumulation absorbs ranker-specific biases that do not transfer.

• Training shows diminishing returns. Performance rises quickly in early batches and then plateaus as more training queries are added; downside risk declines overall but not monotonically.

• Modify operations are targeted repairs, not indiscriminate expansion. Of nine under-performing behaviors that were modified, eight improved beyond a matched mean-reversion control (sign test, p = 0.020), with an average adjusted increase of 5.3 percentage points in helpful rate. Modifying already-above-average behaviors produced no measurable gain.

• Knowledge transfers to unseen categories. In leave-one-category-out evaluation, handbooks trained without a target category retained a substantial portion of the fully trained model's improvement, sometimes surpassing it; the overall MRG decrease was 0.07.

• A traceable failure-to-fix case study. For the CableCreation 6 FT Flat 3.5mm Aux Cable, an existing durability behavior degraded rank from 4 to 5 (gain of −1); after a Modify (B012) and an Add (B016) were committed, rank moved from 4 to 0 (gain of +4). B016 was then applied in 185 subsequent episodes with a 62% helpful rate, 13% harmful rate, and mean gain of +1.90.

• Backbone sensitivity. Rerunning the full procedure per backbone gave positive rank gains for all three models, with different trade-offs: GPT-5.4 (MRG 1.611, handbook size 47), Qwen3.5-122B-A10B (MRG 1.507, size 29), Gemini-3.5-flash (MRG 1.700, size 46).

Methodology in Plain English

The researchers treat optimization as a repeating cycle of deployment and evolution, run over batches of query-product pairs.

The memory unit. Instead of a free-form instruction, knowledge is stored as individual behaviors. Each one says what edit to make (for example, name the material at the likely wear point), which fields it may be applied to, which other behaviors it conflicts with, and how often it has helped or hurt. A behavior's score is simply its helpful count minus its harmful count.

Deployment. In each iteration, an LLM editor is handed the current active behaviors along with a query and a product listing. It picks relevant behaviors, rewrites only the fields those behaviors permit, and reports which behaviors it used. A generative ranking model then re-ranks the full candidate set, and the change in the target product's position is recorded as the episode's rank gain (positive means it moved up).

Evolution. After all 100 episodes in a batch finish — keeping the handbook fixed within a batch — the outcomes are aggregated. Every behavior applied in an episode receives credit or blame for that episode's direction, since individual contributions cannot be isolated. Three components then update the handbook: the Reflector reads recent episodes and proposes structured additions or modifications (it never deletes); the Moderator rejects proposals that are too similar in embedding space to existing behaviors (cosine similarity above 0.9, using all-MiniLM-L6-v2) and then uses an LLM to check for functional overlap; the Curator commits survivors with stable identifiers and updated versions.

Training and testing. Training starts from a seed handbook of nine general GEO heuristics and runs 1,000 training queries as ten batches of 100, using GPT-5.4 for all LLM components. Evaluation uses a disjoint, category-balanced set of 900 queries (150 per category) from the E-GEO benchmark, which pairs Reddit r/BuyItForLife shopping queries with Amazon listings; after the final training batch the handbook is frozen, and no statistics update during evaluation. Four metrics are computed: Mean Rank Gain and Success Rate for effectiveness, Downside Risk (squared penalties on rank drops) and Category Variance for risk and consistency.

Why This Matters

Impact on research. The paper makes a concrete case that agentic memory should be structured and scoped rather than a growing pile of prompt text. Its finding that unmoderated accumulation inflates the handbook while hurting consistency — and that it transfers worse to unseen rankers — is a useful empirical caution for the broader self-improving-agent literature, which has mostly studied open-ended reasoning, coding, and planning rather than field-structured documents scored by indirect relative rank feedback.

Real-world applications:

• E-commerce listing optimization. Sellers and marketplace operators could apply a frozen handbook to rewrite titles, feature bullets, and descriptions for visibility in AI-generated shopping answers.
• Interpretable catalog auditing. Because each edit is tied to a named behavior with a track record, teams can inspect exactly which tactics are being applied and which have historically backfired.
• Cross-platform deployment. A handbook evolved against one ranking engine can be reused against others without retraining, as the cross-ranker experiments suggest.
• Cost-efficient inference. The compiled handbook functions as a static rule set applied in a single pass, with no iterative optimization loop at inference time.

Industry relevance. The work originates from LG Electronics' AI Lab and targets a live commercial problem: as LLM-based shopping assistants displace conventional search, visibility depends on how a generative engine interprets listing text. The paper's emphasis on downside risk and category variance speaks directly to production concerns about rewrites that damage rankings, and its moderation mechanism offers a cheap safeguard. The authors also flag a compute-access concern: evolving handbooks with high-capacity models favors better-resourced organizations, and distilling behaviors into smaller models is suggested as a mitigation.

Future Directions

• Disentangling transferable knowledge from ranker-specific artifacts. Because the editor, ranker, Reflector, and Moderator were all instantiated from GPT-5.4, learned behaviors may partly reflect that model's preferences; the cross-ranker probe is a partial check, and full separation remains open.

• More precise credit assignment. Each episode yields one rank change for an entire rewrite, so outcomes are attributed to all applied behaviors, and a good behavior can absorb harmful counts from co-applied ones. Per-behavior counterfactual rewrites are suggested but noted as substantially more expensive in ranking calls.

• Reversibility and faster reaction to bad edits. Modified behaviors inherit their accumulated counts, so the active-status criterion reacts slowly to a harmful modification, and GLOBE currently has no mechanism to revert such a revision.

• Temporal and multimodal extension. Real e-commerce ranking algorithms change continuously, so accumulated behaviors may degrade and need periodic re-evolution; extending text-only field-scoped constraints to multimodal listings is named as an open avenue.

Target Audience

This paper is most useful to researchers and practitioners working on LLM agents, agentic memory, and generative engine optimization, as well as e-commerce search and catalog teams evaluating how AI shopping assistants rank products. It also suits readers interested in self-improving systems that need interpretable, auditable knowledge rather than opaque prompt updates. A working understanding of LLM prompting and ranking metrics is helpful; the methods themselves are described in accessible terms with formulas kept simple.

Authors’ abstract

Optimizing product listings for generative search requires language agents not only to rewrite effectively but also to continually self-improve by retaining successful strategies across diverse categories. However, existing approaches often treat structured listings as free-form text, leaving effective strategies implicit in prompts, search trajectories, or rewritten outputs rather than accumulating them as a reusable agentic memory. To address this limitation, we introduce GLOBE, an agentic framework that drives continual self-improvement by storing optimization knowledge in an evolving handbook of explicit, field-scoped behaviors. In each episode, an Editor applies relevant behaviors and receives rank-based feedback, which a Reflector converts into behavior updates. Crucially for dependable execution in real-world settings, a Moderator removes redundant proposals, a Curator commits accepted changes, and consistently harmful behaviors are deactivated. This cycle transforms episodic feedback into a persistent, field-scoped optimization memory. Experiments show that structured behaviors outperform monolithic prompt optimization, while feedback-driven evolution and moderation further improve effectiveness and reduce downside risk. The learned behaviors also remain effective on unseen product categories, suggesting that agentic memory evolution supports interpretable and transferable product optimization.