QD-Rec: Open-Ended Evolution of Recommendations via Quality-Diversity

AI summary

Overview

Research area: Recommender systems and quality–diversity (QD) search, specifically applying open-ended evolutionary search with foundation models to decision-making over a fixed pool of candidates.

Technical level: Advanced. The paper combines Bayesian contextual bandits (linear Thompson sampling), archive-based QD methods (MAP-Elites and novelty search lineage), and LLM-driven proposal and judging loops; readers will benefit from familiarity with bandit exploration–exploitation trade-offs and QD terminology.

Scope: The paper formalizes "open-ended recommendation" as a search over item–hypothesis pairs drawn from a fixed candidate pool, and introduces QD-Rec, an archive-based method that pairs bandit-driven item selection with LLM-generated natural-language justifications, evaluated across four domains with two proposer models.

What This Paper Is About

Open-ended search with foundation models has produced results such as faster algorithms and new mathematical constructions, but it has not been applied to recommendation for decision-making. In many high-stakes decisions, the set of options is fixed (the companies that exist, the papers already written, the customers already known), yet the reasons for choosing an option are not fixed: the same item can be justified in many different ways. The paper asks whether QD search can build a diverse, high-quality set of recommendations, each consisting of an item together with a natural-language hypothesis explaining why it fits a specific decision maker's profile, rather than returning a single ranked list.

Key Contributions

1. A formal problem statement. The authors define open-ended recommendation as returning a set $\mathcal{S} \subset \mathcal{C} \times \mathcal{H}$ of item–hypothesis pairs from a fixed candidate pool $\mathcal{C}$, where the hypothesis space $\mathcal{H}$ is unbounded and evaluation is costly (quality is known only after the hypothesis is written and judged), under a budget of at most $T$ evaluations.

2. The QD-Rec method. An archive-based QD search combining three components: a diversity-aware contextual bandit (Bayesian linear regression with linear Thompson sampling plus a nearest-neighbor cosine-distance bonus) for item selection; an LLM "proposer" that writes hypotheses conditioned on archive exemplars using mutate, crossover, or explore operators; and a two-stage archive admission rule based on local novelty checks and blind pairwise duels.

3. Two ablations isolating components. Novelty (archive mechanics only, uniform-random items, no exemplars), Novelty+Evo (adds the evolutionary few-shot operator), and QD-Rec (adds diversity-aware bandit item selection), compared under matched budgets and shared hyperparameters.

4. Empirical evaluation on four domains and two proposers. Goodreads book recommendation (a grounding check with real human star ratings), M&A target discovery, marketing/customer targeting, and paper-abstract exploration, with claude-sonnet-5 and claude-haiku-4-5 as proposers and gpt-5.6-terra as judge.

Main Findings

• Highest quality in most LLM-judged settings. QD-Rec gives the best mean Quality in most LLM-judged configurations and is never significantly worse than the best variant. The gains are largest on M&A; for example, on the Walt Disney Co profile with the claude-sonnet-5 proposer, QD-Rec scores 2.91 ± 0.05 versus 2.51 ± 0.07 for Novelty+Evo and 2.40 ± 0.12 for Novelty, and on JPMorgan Chase it scores 3.04 ± 0.04 versus 2.54 ± 0.08 and 2.88 ± 0.04.

• Generally the largest archive. QD-Rec builds the largest archive in most settings (for Disney with claude-sonnet-5, n = 214 ± 2 versus 135 ± 5 and 85 ± 1), so its higher Quality does not come from a smaller, more selective archive.

• Diversity is best or tied in most domains. Semantic Diversity is best or tied for best in every domain except M&A, where it is slightly lower.

• Goodreads behaves differently. Where Quality is the reader's own 1–5 star rating plus a human-judged variant, the three variants reach similar mean ratings (e.g., 4.17, 4.15, 4.15 on series/YA with claude-sonnet-5), while QD-Rec matches or beats the ablations on archive size, Diversity, and coverage.

• Quality rises through the run; Diversity saturates. QD-Rec's Quality keeps rising and ends highest, and archive size keeps growing, but Diversity changes little after roughly the first 100 evaluations — attributed to admission being decided by LLM novelty checks against nearest neighbors rather than an embedding-distance threshold.

• QD variants beat training-free re-rankers on human-rated quality. Against relevance-MMR and a quality-weighted DPP with greedy MAP inference on Goodreads, no setting of either baseline's relevance–diversity weight reaches any QD variant on Quality; they approach QD-Rec's Diversity only at their most diversity-favoring settings. The baselines see no ratings, so the gap reflects rating feedback plus the archive, not item selection alone.

• Results hold under a different embedding model. Re-running one profile per domain with Qwen3-Embedding-0.6B truncated to the same 512 dimensions leaves the best variant on Quality and archive size unchanged on all four profiles, though Diversity is higher for every variant because those embeddings are more evenly spread.

• Cost. QD-Rec costs the same as Novelty+Evo, and both cost about 1.3× Novelty; more than 90% of the difference comes from the proposer, since exemplars roughly triple its input. A single run costs between about $13 (Novelty on Goodreads with claude-haiku-4-5) and $62 (QD-Rec on Marketing with claude-sonnet-5).

• One weaker-proposer exception. On ML Papers (OpenEndedness) with claude-haiku-4-5, Novelty+Evo scores higher Quality (5.03 ± 0.07) than QD-Rec (4.87 ± 0.02), and QD-Rec's coverage (0.30 ± 0.01) and archive size (353 ± 34) trail Novelty+Evo (0.26 ± 0.02, 247 ± 31).

Methodology in Plain English

The system runs a fixed number of evaluation cycles and keeps an archive of accepted recommendations, each storing the item, the hypothesis, an embedding of the pair, and its quality score.

Choosing what to evaluate next. Each candidate item has a precomputed embedding (from a paper's title and abstract, a book's title and description, a company's business description, or a customer profile). The method fits a Bayesian linear regression from these embeddings to reward, trained on every score observed so far and warm-started from a seed archive. At each step it samples a plausible reward model (linear Thompson sampling) and picks the item maximizing the sampled quality estimate plus a bonus equal to the cosine distance from the nearest item already in the archive — so items similar to what is already covered are penalized. A weight $\beta = 1$ controls this trade-off; $\beta = 0$ would reduce it to a quality-only bandit.

Writing the hypothesis. The selected item, the decision-maker's profile, and three exemplars sampled from the archive are given to the proposer LLM, which is instructed to mutate one exemplar's angle, cross over several, or explore an angle the exemplars miss. The archive thus acts as an evolving population of reasons.

Deciding what enters the archive. The new pair's three nearest archive neighbors are retrieved by cosine similarity. An LLM judge first answers a binary novelty question; if novel, the pair is admitted immediately with no comparison. If not novel, only the single nearest neighbor is contested in a blind pairwise duel where both entries are anonymized and pointwise scores withheld — if the newcomer wins it replaces that neighbor, otherwise it is discarded. Separating novelty from quality keeps the retention comparison blind and skips the comparison call whenever novelty suffices.

Parallelization and metrics. Runs proceed in generations of up to 20 parallel evaluations against an archive snapshot, with near-duplicate additions filtered and replacement collisions resolved between generations. Four metrics summarize each run's final archive: Quality (mean score; judge score 0–10 or Goodreads star rating 1–5), Diversity (one minus mean pairwise cosine similarity), Coverage (occupied cells on a 25×25 grid over a 2D PCA projection with shared axis bounds padded by 1%), and archive size.

Experimental protocol. Each configuration runs 1,000 evaluations per method across all four domains, repeated over five random seeds, with gpt-5.6-terra as judge in every configuration to avoid self-preference bias (it is a different model family from both proposers). The judge uses reasoning effort "high," as does claude-sonnet-5; claude-haiku-4-5 does not support that parameter. Prior precision is $\lambda = 1$, noise standard deviation $\sigma = 0.1$, $k = 3$ nearest neighbors, three exemplars, and 10 seed entries. Candidate pools are children's books a reader has rated (Goodreads, Wan and McAuley 2018), 6,332 US companies from LSEG Workspace, 10,000 synthetic California customer profiles from NVIDIA Nemotron-Personas-USA, and 17,511 papers accepted at ICLR 2026, ICML 2026, and NeurIPS 2025.

Why This Matters

The work argues that for consequential choices, decision makers benefit from comparing a set of well-argued options over different rationales, not from one ranked list. It shows that the "outer loop" — deciding which candidate to evaluate and which pairs to retain — is itself a meaningful lever, even when the inner hypothesis generator is a single prompted LLM call.

Real-world applications:

• Corporate development / M&A deal sourcing: generating a diverse set of strategic acquisition theses (capability acquisition, market access, vertical integration) rather than a single shortlist of targets.
• Marketing and sales targeting: producing a map of distinct targeting angles over a customer base instead of a ranked list of leads.
• Research support: surfacing diverse, well-justified reading directions across a large literature under limited time.
• Consumer recommendation with explanations: grounded here in book recommendation against real human ratings, indicating the approach survives contact with a non-LLM quality signal.

Industry relevance: The method reuses existing candidate catalogs and item embeddings, adds a bandit that runs locally at no API cost, and keeps total run cost in the $13–$62 range per 1,000 evaluations — a plausible budget for exploratory decision support. Its explainable-recommendation framing (each item paired with a reason) also fits settings where a recommendation must be defended to a stakeholder.

Future Directions

• Stronger reward models. Replace the linear regression over fixed item embeddings with neural contextual bandits to capture nonlinear relationships, or discard the parametric bandit entirely for a coding agent that reads the archive, reasons about which regions look exhausted or promising, and writes its own inspectable selection rule.
• Smarter parent selection. Exemplars are sampled uniformly from the archive; selecting them by quality, recency, or how often they have already served could help, and bandits could be applied to parent selection in addition to item selection.
• Richer inner loop. The hypothesis generator here is a single prompted LLM call; substituting a full research agent is expected to compose with the outer-loop machinery.
• Explaining why Diversity saturates. Diversity plateaus after roughly the first 100 evaluations while archive size keeps growing, plausibly because admission is governed by LLM novelty judgments rather than embedding-distance thresholds — an open question about how admission rules shape archive geometry.

Target Audience

Researchers and practitioners working on LLM-driven search and evolutionary optimization (QD, MAP-Elites, novelty search, FunSearch-style loops) who want to see the paradigm extended to selection from fixed candidate pools; recommender-systems researchers interested in diversity, explainability, and bandit-based exploration; and applied teams in deal sourcing, marketing, or research tooling who need diverse, justified option sets rather than ranked lists. The paper is written as a first work in this line and keeps components simple, so it is accessible to graduate students familiar with bandits and embeddings, though the full method requires comfort with Bayesian linear regression and QD archive mechanics.

Authors’ abstract

Open-ended search combined with foundation models has produced advances such as faster algorithms, new results on open mathematical problems, and self-improving agents. We extend this paradigm to recommendation for decision-making, focusing on decisions where the set of possible choices is fixed but the reasons for choosing them are not. In high-stakes decisions, e.g., which company to acquire or which customer to target, comparing diverse, well-reasoned options can be more useful than a single ranked list. We formalize open-ended recommendation as building a diverse, high-quality set of recommendations, each an item together with a hypothesis, a natural-language reason why the item fits the needs of a decision maker. We propose QD-Rec, a quality--diversity method that selects items with a contextual bandit and generates hypotheses with a foundation model from earlier ones kept in an archive as stepping stones. A new recommendation enters the archive if it is novel or better than its nearest neighbor. Across M&A, customer targeting, and paper and book recommendation, QD-Rec gives the highest mean quality and builds the largest archive in most settings.