Hybrid Generative-Tagging Decoding for Efficient Product Attribute-Value Extraction

AI summary

Overview

Research area: Natural Language Processing, specifically structured information extraction for e-commerce (product attribute–value extraction) and efficient LLM inference/serving.

Technical level: Advanced. The paper assumes familiarity with autoregressive language models, prefill/decode separation, span-pointer heads, and serving stacks such as vLLM.

Scope (one sentence): The paper proposes a single-model decoding architecture that shares one LLM prefill between a fixed-vocabulary span-tagging head and an autoregressive generative head, routing copyable product values to tagging and only non-copyable values to generation.

What This Paper Is About

E-commerce catalogs need accurate attribute–value pairs (e.g., brand: Mrs Ball's, cuisine: South African), but the two dominant extraction approaches are complementary rather than sufficient on their own: tagging models are fast and grounded in the source text but can only copy spans from a fixed label space, while generative LLMs handle normalization, implicit values, and rare attributes but must decode every value token by token, which is slow at catalog scale and tends to drop attributes buried deep in long decoded lists.

The paper's goal is to get both behaviors from one model and one prefill: copy verbatim values out of the source with a lightweight span head, and reserve autoregressive decoding only for the "remainder" — attributes requiring reformatting, implicit values, and attributes outside the tagging vocabulary. In the authors' corpus, 81.1% of values are taggable and 18.9% are generative, and the two types co-occur within most records.

Key Contributions

1. Hybrid generative–tagging decoding. A decoding strategy in which a single LLM prefill feeds two decoupled channels — an autoregressive generative channel and a tagging channel — so that copyable values are extracted with zero decoding steps while generation is reserved for implicit and out-of-vocabulary values. The two channels are merged by attribute name, with the generative value preferred on collision.

2. A shared span head over 500 frequent attributes. Each attribute is represented by the base model's own mean-pooled encoding of its name string; a cross-attention block produces a contextualized query, a start pointer scores all prompt positions plus a learned NULL slot (the abstention/presence decision), and an independent end pointer delimits the value within a forward window of width B = 96. All K = 500 attributes are resolved in one batched, non-autoregressive pass over prefill states.

3. A routing-based training scheme. Gold in-vocabulary pairs whose values occur as contiguous token-exact spans are routed to the tagging target and removed from the generative target; everything else stays generative. The loss is L = L_gen + L_tag, where L_tag averages a span loss that marginalizes over all valid occurrences of the value and a NULL loss over 30 sampled negatives per example.

4. An efficiency and coverage result demonstrated end to end. Integrated into vLLM, the hybrid raises attribute coverage by 10% and extracts 63% more correct attribute–value pairs at no precision cost while reducing latency by 44% relative to a generative baseline.

Main Findings

• Recall gain without precision loss on the Amazon catalog. Hybrid recall is 80.2%, versus 69.8% for the generative baseline and 65.3% for the tagging-only model. Hybrid precision is 97.1%, statistically tied with the generative baseline's 97.5%, even though the hybrid emits 63% more pairs per record (34.8 vs. 21.3). Equivalence is supported by record-level percentile bootstrap intervals (10,000 resamples) placing all three systems within a 0.8-point band, and a TOST equivalence test at α = 0.05 bounding pairwise differences within ±1 point.

• The gain concentrates on extractive attributes. On the extractive subset, the hybrid covers 82.5% of gold attributes against 70.2% for the generative baseline. Of the gold slots the hybrid covers, 15.7% are reached only by the tagging channel, 16.6% only by the generative channel, and 64.4% by both.

• Blinded human quality ratings favor the hybrid. Mean 1–5 ratings are 4.40 ± 0.43 (hybrid), 3.93 ± 0.57 (generative), and 3.50 ± 0.51 (tagging) over 1,000 records. The hybrid ties or beats the generative baseline on 70.2% of records and is the strict top-rated system on 50.9%. Paired Wilcoxon tests give mean differences of +0.46 (z = 11.9, p < 10⁻³¹) over generative and +0.90 (z = 22.2, p < 10⁻¹⁰⁸) over tagging.

• Speed and output length both drop. Mean wall-clock time falls from 56.6 ms to 31.5 ms per record (a 1.80× speed-up), while mean generated length drops from 257.8 to 80.0 tokens (−69.0%). The tagging-only model runs at 9.9 ms (5.69×), and cost decomposition shows the hybrid's 31.5 ms splits into 9.9 ms for shared prefill plus the span head over all 500 attributes and 21.6 ms for the generative remainder.

• Strong grounding, with fabrication concentrated in the generative channel. 90.8% of hybrid-emitted values occur verbatim in the source; the tagging channel is 100.0% verbatim. On extractive-type attributes, a format-insensitive substring check flags 3.0% of generative values and 1.3% of hybrid values, but a 250-case manual audit (125 per system) estimates genuine fabrication at ≈0.12% for generative and ≈0.04% for the hybrid — a difference the authors explicitly do not claim is statistically significant.

• Public benchmarks: competitive but not dominant. Using Slot F1, the hybrid reaches 0.841 on AE-110K (generative 0.838, tagging 0.830, MVP-RAG 0.836, GenToC 0.705, HPD 0.839) at 62.4 ms (2.1×), and 0.826 on OA-Mine (generative 0.810, tagging 0.818, MVP-RAG 0.826, GenToC 0.775, HPD 0.876) at 114.7 ms (2.2×) — trailing HPD by about 5 points on OA-Mine, though HPD assumes access to the full per-category attribute schema and cannot run under optimized stacks such as vLLM.

• Gains hold across backbones and model sizes. On Qwen3-1.7B, Qwen3-4B (LoRA), and Gemma-3-1B, the hybrid improves recall over the generative baseline by +9.4 to +13.0 points, keeps precision within a point, and delivers 1.30×–1.80× speed-ups. A stronger generative model does not close the recall gap.

• Span-head design matters for precision. Per-token tagging reaches 87.7 precision / 74.1 recall / 36.3 pairs per record; an anchored pointer head reaches 93.1 / 71.5 / 34.3; the chosen independent-pointer head reaches 97.2 / 70.5 / 34.8.

• Exact-match scoring preserves the ranking but understates precision. Under exact match, hybrid precision is 63.8 and F1 77.0, versus 54.5 / 69.9 for generative and 56.1 / 71.2 for tagging — far below the ~97% LLM-judged figures because exact match penalizes correct reformatting.

• Vocabulary size acts as a speed/coverage dial. Sweeping K over {100, 250, 500, 1000} moves coverage from 70.0% to 92.0%, recall from 69.2 to 80.9, and latency down from 37.5 to 29.0 ms; recall plateaus beyond K = 500 (only +0.7 from doubling to 1000).

Methodology in Plain English

The system begins with one pass of a small LLM (Qwen3-0.6B by default) over the product text, producing hidden states for every input position. Both extraction channels read those same hidden states, so the expensive part of the computation is paid once.

For the tagging channel, the authors take the 500 most frequent attribute names in their corpus — a vocabulary covering 87.8% of attribute instances — and describe each one using the model's own encoding of the attribute-name string. A small cross-attention block lets each attribute "look at" the whole product text. Two scoring functions then act like two pointers: one scans every position and asks "could the value start here, or is this attribute simply absent?" (the absence option is a learned NULL slot), and a second independently scores where the value ends, constrained to a window of 96 tokens after the chosen start. Because the pointers read pre-stored hidden states, all 500 attributes are resolved in one parallel batch with no token-by-token decoding, and the identified text is copied verbatim as the value.

The generative channel is an ordinary autoregressive decoder that emits semicolon-delimited name: value; pairs, but it is only trained and asked to produce the leftover attributes: those outside the 500-name vocabulary, those whose values are implicit or need reformatting, and in-vocabulary attributes whose value does not appear cleanly in the text. Training targets come from a simple rule — search the source for an exact token-level match of the gold value; if found, supervise the span head and delete the pair from the generative target; if not, leave it to the generator. A NULL loss over sampled negatives (including hard negatives, i.e., real attributes whose values are not copyable spans) teaches the head to abstain rather than copy something semantically related but wrong.

At inference in vLLM, the two channels run off one prefill and their outputs are merged by attribute name. The authors evaluate with an LLM judge (GPT-5.6 Luna) rather than exact string matching, because normalization and formatting differences would otherwise be scored as errors; on a manual audit of 4,795 pairs, human judgment agreed with the judge on 99.04% of pairs (κ = 0.867).

Why This Matters

Impact on research. The paper reframes extraction efficiency as a routing problem rather than a decoding-speed problem: instead of making each generated token cheaper (speculative decoding, paged attention), it removes tokens from the autoregressive path entirely. The authors note this is orthogonal to and stackable with serving-level optimizations, and they show the decomposition also fixes a recall failure mode of long generative lists that scaling the backbone does not fix. It also offers a clean contrast with GenToC, which mixes generation and tagging for noisy-label robustness rather than for concurrency and efficiency.

Real-world applications:

• Catalog cleanup at marketplace scale, where merchant-supplied fields are noisy, incomplete, or missing and manual repair does not scale — the motivating setting of the paper, over 111K Amazon US records and 2.62M attribute–value pairs.
• Product matching and deduplication, which depend on structured attribute pairs across listings.
• Agentic web navigation and conversational shopping assistants, cited by the authors as consumers of the structured knowledge base.
• Any high-throughput span-plus-normalization extraction pipeline where most values are copyable but a minority need reformatting — the 81.1% taggable / 18.9% generative split is the design premise.

Industry relevance. The speed-up (56.6 ms → 31.5 ms per record, 1.80×) matters in direct proportion to catalog volume, and the 63% increase in extracted correct pairs at tied precision translates into catalog completeness without a precision regression. Because the hybrid keeps the standard decoding path, it retains compatibility with vLLM-style serving — a practical advantage the authors explicitly contrast with HPD, which rewrites decoding logic. The backbone ablation showing gains from 0.6B to 4B parameters and across two model families suggests the benefit is not tied to one model choice.

Future Directions

• Improve span-head quality on locatable values. The error decomposition shows 82.4% of over 97K in-vocabulary instances are span-routable, and among the hybrid's errors 70.5% are true span errors (wrong occurrence or boundary) versus 29.5% span-unwinnable cases — making span-head accuracy the clearest remaining lever.

• Extend beyond text-sourced attributes from a single service. The limitations section scopes out image-sourced attributes, other catalog domains, and other languages, and states that cross-domain transfer is not evaluated, so the reported gains may not hold for catalogs with different attribute distributions or text conventions.

• Address annotation and evaluation noise. Training and evaluation data are LLM-annotated (by Kimi-K2.5), and precision and quality metrics come from an LLM judge, so both inherit model biases — a cleaner human-labeled benchmark would test the reported precision tie more independently.

• Push vocabulary and routing policies further. The K sweep plateaus at K = 500, but coverage still rises to 92.0% at K = 1000; better routing decisions (which attributes are worth tagging versus generating) and larger or adaptive vocabularies are an open design space.

Target Audience

Applied NLP and ML engineers building production extraction systems, particularly those working on e-commerce catalogs, who already understand transformer prefill/decode mechanics and LLM serving. It is also relevant to researchers interested in hybrid discriminative–generative architectures, span-pointer/query-based tagging, and inference-cost reduction that composes with existing serving stacks. Readers looking for a beginner-level introduction to information extraction, or for multilingual and multimodal coverage, will find the paper out of scope on those axes.

Authors’ abstract

Attribute-value extraction is important for building high-quality product catalogs in e-commerce. Existing systems either rely on generative or tagging strategies for extracting the attribute names and their corresponding values. Tagging models directly label spans in the source, making them fast and grounded, but they cannot produce values that require reformatting or generate rare attribute names outside their label space. Generative models avoid these drawbacks by generating both attribute names and values token by token, but this auto-regressive nature makes them slow and limits attribute coverage. We introduce hybrid generative-tagging decoding, a novel model architecture that combines tagging source spans and open-ended generation. For frequent attributes whose values are copyable, a tagging head extracts values directly from the source. This lets the generative head focus on the remainder - rare attributes outside the tagger's label space and non-copyable values, including implicit or normalized values. Integrated into vLLM inference, the hybrid decoder raises attribute coverage by 10% and extracts 63% more correct attribute-value pairs at no precision cost, while reducing latency by 44%.