Attribute-Aware Controlled Product Generation with LLMs for E-commerce

AI summary

Overview

Research area: Natural Language Processing for e-commerce — synthetic data generation with Large Language Models (LLMs) for product attribute extraction.

Technical level: Intermediate. The pipeline is described conceptually and in algorithmic pseudocode, but assumes familiarity with LLM prompting, fine-tuning, and attribute-value extraction tasks.

Scope: The paper describes a controlled LLM-based pipeline that modifies existing product listings to create labeled synthetic training data (correct, incorrect, and unknown attributes), then validates it with human annotators and downstream attribute-extraction experiments on the MAVE dataset.

What This Paper Is About

E-commerce systems need large, high-quality labeled datasets that link structured product attributes (such as color or material) to the free-form text where those attributes appear, but manual annotation at scale is impractical and expensive. The authors build a framework that takes existing products and uses LLMs with attribute-aware prompts to deliberately modify one attribute at a time — sometimes correctly, sometimes introducing a subtle contradiction, sometimes removing all mentions of it — while keeping the listing coherent and marketplace constraints intact. The goal is to produce synthetic training data that works as well as real annotated data when training attribute extraction models.

Key Contributions

1. A controlled modification framework with three generation strategies: attribute-preserving (positive) modification, controlled negative example generation, and systematic attribute removal — sampled with probabilities π_c = 0.5, π_i = 0.25, and π_u = 0.25.
2. A multi-step generation architecture combining a Value Provider LLM (Claude Haiku), a Similarity LLM (sentence-transformers all-MiniLM-L6-v2) for selecting semantically distinct negative values, and a Generation LLM, all driven by structured prompts (ROLE ⊕ INSTRUCTION ⊕ CONTEXT ⊕ FORMAT).
3. Marketplace-aware constraints built into the prompts: brand anonymization (e.g., replacing a real sport brand with "AthleteX"), preservation of original text structure, cross-field attribute consistency, multilingual support, and store-specific formats such as imperial units in US listings.
4. Human and downstream validation at scale: expert evaluation of 2,000 synthetic products across six quality aspects with three annotators per product and majority voting, plus attribute extraction experiments with FLAN-T5-base comparing synthetic-only, original-only, and three hybrid data mixes.

Main Findings

• Human-rated naturalness and validity: Of 2,000 synthetic products, 99.6% were rated as natural e-commerce language, 96.5% as containing valid attribute values, and 95.8% as having successful brand modification.
• Consistency by generation type: Attribute consistency across title, description, and features was 94.2% for correct examples, 93.0% for incorrect examples, and 88.3% for unknown-attribute examples.
• Content preservation: 88.8% of products showed no unintended changes, 7.0% showed acceptable changes, and 4.2% required major revisions, "primarily in cases with empty descriptions."
• Empty descriptions handled: The model generated appropriate content for all 47 cases with empty descriptions (100%).
• Downstream extraction performance: On the MAVE test set with FLAN-T5-base, synthetic-only training reached 60.48% accuracy versus 60.79% for original data only and 13.40% for the zero-shot baseline.
• Hybrid data helps most: The best configuration was 75% original + 25% synthetic at 68.82% accuracy; increasing the synthetic share reduced accuracy to 66.11% (50/50) and 64.44% (25/75) — decreases of 2.71 and 4.38 percentage points respectively.
• Apparent errors are often format differences: Manual review of 441 of 959 initially incorrect predictions in the synthetic-only configuration found seven categories of valid variation, including granularity differences ("running shoe" vs "running"), morphological variation ("wall sticker" vs "wall stickers"), missing units ("1200" vs "1200 thread count"), and contextual synonyms ("striped" vs "stripe").
• Text distribution is close to the original: Type-Token Ratio was 0.89 original vs 0.88 synthetic for titles, 0.82 vs 0.81 for descriptions, and 0.85 vs 0.84 for features; semantic similarity (cosine) was 0.84 (title), 0.85 (description), and 0.93 (features); KL divergence was highest in titles (1.12) and lowest in features (0.24).
• Source errors propagate: Validity and consistency errors traced back mainly to the source dataset, particularly overly generic attribute values (such as "type") and semantically misaligned attributes.
• Cost profile: Two LLM calls per product — a Value Provider call averaging 402 input tokens and 10 output tokens, and a generation call adding 1,480–1,600 input tokens and around 141 output tokens. Using Claude Haiku's published pricing ($0.80 per million input tokens, $4.00 per million output tokens), this is compared against human annotation costs of approximately $0.11 per 50 tokens; generation completes in hours rather than weeks.
• Reported dataset scale differs between sections: The abstract and introduction describe MAVE as 2.2 million products across 1,257 categories, while Appendix A describes it as over 3.3 million products across 1,212 categories with 662 unique attributes (2.1M positive, 1.1M negative, 1.38 attributes per product on average); Appendix A.3 and Table 4 report 1.41 attributes per product (std dev 0.81), 1.31 evidence spans per attribute (std dev 2.00), and 8.58 paragraphs per product (std dev 3.38). The paper does not reconcile these figures.
• Attribute distribution is long-tailed: "Type" is the most common attribute at 25.1% (1,005 products), followed by "Special Occasion" and "Style" at 5.8% each; the top 10 attributes cover 63.1% of annotations.
• Evidence is spread across fields: Evidence spans occur in Features (57.9%), Description (15.3%), Title (11.7%), Brand (10.0%), and Price (5.1%).

Methodology in Plain English

The researchers start from a real product listing that has both structured attributes (key-value pairs like {color: "red"}) and free text (title, description, features). For a given product, they pick one category-relevant attribute. A first LLM proposes a new value for that attribute; in the negative-example case, they first generate a pool of values at temperature T = 1.0 and then use a similarity model to pick one that is semantically far from the correct value. A generation strategy is then sampled: 50% correct modification, 25% incorrect, 25% removal. A second LLM call rewrites the listing according to a structured prompt that assigns it the role of an e-commerce expert, gives it the product context and modification target, requires brand anonymization and structure preservation, and asks for JSON output. The result is a complete synthetic listing where the attribute is either faithfully reflected, subtly contradicted, or absent.

To check quality, the authors had expert annotators judge 2,000 synthetic products, showing each annotator the original and synthetic versions side by side with changes highlighted, with three labels per product and majority voting across six aspects (attribute value quality, negative example coherence, cross-field consistency, brand modification, content preservation, professional writing). For downstream evaluation, they fine-tuned FLAN-T5-base with a 512-token context window, framing attribute extraction as a generation problem, using ~800 correct examples for training (80%) and validation (20%) and evaluating on the remaining ~1,000. Training used AdamW at a learning rate of 5e-5 for up to 12 epochs with early stopping; generation used beam search with 5 beams, temperature 0.7, top_k 50, top_p 0.95, and max_length 20. Six data configurations were compared: zero-shot, original only, synthetic only, and 75/25, 50/50, and 25/75 original-to-synthetic mixes.

Why This Matters

Impact on research. The paper argues it is the first work to address comprehensive synthetic product information generation for e-commerce, moving beyond single-aspect generation approaches such as FPDG that rely on explicit entity labeling and architectural constraints. It offers an LLM-based alternative that uses semantic understanding instead, and it documents how annotation inconsistencies between a model and a public dataset's gold labels can create the appearance of error.

Real-world applications:

• Bootstrapping attribute extraction models for new categories or new marketplaces where no labeled data exists yet.
• Generating balanced training sets that include negative and "unknown" examples, which real catalogs rarely label at scale.
• Cost-effectively expanding coverage for low-resource product categories, replacing or supplementing manual content-quality annotation.
• Creating multilingual or store-specific variants of the same product data through prompt-level constraints such as unit systems.

Industry relevance. The paper cites that large e-commerce services spend an average of $12.9 million annually on content quality assurance, and that noisy or inaccurate product details are common. It also reports successfully regenerating the complete MAVE dataset, which speaks to scalability for catalog-scale deployment.

Future Directions

• Evaluate on non-positive examples. Current downstream extraction experiments use only correct examples; the authors call for testing models trained with incorrect and unknown attributes on more diverse downstream tasks.
• Extend beyond a single attribute. The current implementation modifies one attribute at a time; future work would handle multiple attributes while keeping their interactions consistent.
• Stronger controls on attribute values. Manual review revealed granularity variation (e.g., "running shoe" vs "running") and distribution mismatch between original and synthetic values; attribute-specific vocabularies or controlled generation could standardize this.
• Better use of hybrid data and stronger modification control. The authors want to understand why hybrid mixes outperform both single sources, explore how mixing ratios interact with attribute types and categories, and improve control over unintended changes beyond the target attribute. They also flag that source descriptions can contain vague values, missing brand mentions, and inconsistent attribute definitions, and that synthetic data sometimes improves on these — which complicates evaluation.

Target Audience

This paper is most useful for applied NLP and machine learning engineers building e-commerce search, filtering, or catalog-quality systems; researchers working on synthetic data generation and data augmentation for information extraction; and product or data teams deciding whether to invest in LLM-based annotation pipelines rather than manual labeling. Readers evaluating cost-benefit tradeoffs between synthetic and human annotation will find the token-level cost breakdown and the hybrid mixing results directly relevant.

Authors’ abstract

Product information extraction is crucial for e-commerce services, but obtaining high-quality labeled datasets remains challenging. We present a systematic approach for generating synthetic e-commerce product data using Large Language Models (LLMs), introducing a controlled modification framework with three strategies: attribute-preserving modification, controlled negative example generation, and systematic attribute removal. Using a state-of-the-art LLM with attribute-aware prompts, we enforce store constraints while maintaining product coherence. Human evaluation of 2000 synthetic products demonstrates high effectiveness, with 99.6% rated as natural, 96.5% containing valid attribute values, and over 90% showing consistent attribute usage. On the public MAVE dataset, our synthetic data achieves 60.5% accuracy, performing on par with real training data (60.8%) and significantly improving upon the 13.4% zero-shot baseline. Hybrid configurations combining synthetic and real data further improve performance, reaching 68.8% accuracy. Our framework provides a practical solution for augmenting e-commerce datasets, particularly valuable for low-resource scenarios.