VOYAGER: A Training Free Approach for Generating Diverse Datasets using LLMs

AI summary

Overview

Research area: Natural Language Processing; synthetic data generation with large language models; diversity optimization for text corpora.

Technical level: Intermediate. Readers need some familiarity with LLM prompting, text embeddings, and basic linear algebra (matrices, determinants, kernels). The Determinantal Point Process machinery is introduced from scratch, but the theoretical section assumes comfort with matrix notation.

Scope: The paper presents Voyager, a training-free, iterative algorithm that uses Determinantal Point Processes and LLM-based prompt refinement to generate synthetic text datasets whose diversity is measurably higher than standard prompting and sampling baselines, demonstrated on four creative writing tasks and two reasoning tasks.

What This Paper Is About

LLMs are widely used to generate synthetic datasets when real-world data is scarce, but the generations tend to be repetitive, semantically clustered, and prone to mode collapse — a problem that temperature and top-p/min-p sampling do not fix because those methods only look at the next-token distribution, not at global similarity across generations. Voyager attacks the problem at the dataset level: it iteratively generates batches of text, keeps only the samples that meaningfully increase the "volume" spanned by the dataset, and uses text-space feedback to steer the model toward unexplored regions — all without touching model weights, so it works with closed-source models.

Key Contributions

1. A principled formulation of diversity maximization. The paper casts dataset diversity as a geometric quantity — the determinant (volume) of the similarity matrix — and connects it to Determinantal Point Processes, providing theoretical justification (Lemma 1 and the maximum volume submatrix reduction) for why the algorithm should work.
2. A training-free method for both open- and closed-weight models. Voyager requires no access to model weights or logits, relying only on prompting and text-space feedback, which makes it compatible with black-box APIs.
3. Strong empirical gains across task types. The authors report diversity improvements of 1.5–3 times over popular baselines, on creative writing and reasoning generation tasks, with no significant degradation in judged quality.
4. A computational efficiency argument. Because the method keeps fixed-size sets of anchors and explorers rather than comparing against the full generation history, the reported cost is far below a naive approach that generates a large universe and then runs a DPP over the whole dataset.

Main Findings

• Voyager beats all baselines on diversity in creative writing. On the four creative tasks (topical sentences, conversation, poem, movie plot), Voyager achieved the best Lexical, Cosine, and Vendi scores in every table. The paper reports an average Vendi score improvement of 2.96 times over Default and 0.43 times over Hierarchical across all creative tasks.

• Voyager also wins on reasoning tasks. On grade-school math question generation and logic puzzle generation, the paper reports an average Vendi score improvement of 4.12 times over Default and 1.02 times over Hierarchical across all reasoning tasks.

• Quality does not collapse. Using an LLM-as-Judge panel, Voyager's quality scores remain comparable to baselines. For example, on the sports sentence task Voyager scored 22.257 ± 2.060 versus Hierarchical's 22.852 ± 0.984 and Default's 21.701 ± 0.745, on a 0–25 scale. On the poem task Voyager scored 24.505 ± 0.905, the highest among all methods.

• Hierarchical prompting is the strongest conventional baseline. The paper states it "very significantly outperforms other methods in this class," which the authors attribute to domain knowledge and explicit topic-level instructions.

• Prompt-based and history-based diversity controls help only modestly. The Diverse-keyword baseline improves over Default and temperature baselines across all metrics; History-based prompting improves only marginally. The authors suggest this is because History enforces non-redundancy with a fixed window rather than global diversity.

• Voyager usually uses fewer LLM calls than the best competing baseline. On the sports task Voyager used 443 LLM calls versus Hierarchical's 550; on the math task 399 versus 550; on the logic puzzle 393 versus 550. On the poem (615) and movie plot (695) tasks Voyager used more LLM calls than Hierarchical's 550 but far fewer than the baseline methods' 50 only because those produce less diverse data.

• Human annotators confirm the automatic metrics. On 200 pairs (100 from Default, 100 from Voyager) across Sports and Math, rated by 3 annotators on a 1–4 dissimilarity rubric, Voyager scored 3.82 ± 0.28 versus Default's 2.16 ± 0.55 on Sports, and 3.72 ± 0.33 versus 1.56 ± 0.36 on Math. Inter-annotator agreement was Fleiss' Kappa 0.41 (Sports) and 0.34 (Math), and Krippendorff's alpha 0.74 (Sports) and 0.72 (Math).

• Diverse explorers matter. Replacing DPP-based explorer selection with random selection (Voyager-RE) on the sports task reduced Vendi score from 14.282 to 11.852, lowered quality from 21.875 ± 1.513 to 21.473 ± 1.881, and raised LLM calls from 252 to 361.

• Textual gradients reduce wasted generations. Disabling prompt refinement (Figure 2) produced a much higher average rejection rate and required more iterations to reach the same dataset size on the sports task.

• Diversity gains are not only an artifact of longer outputs. With a length-controlled prompt ("Generate a poem. Restrict the generated output to 150 tokens"), Voyager still outperformed baselines. In that setting the Default method averaged 131.28 ± 12.761 tokens and scored a Vendi of 3.304 and Lexical of 0.775 ± 0.053.

Methodology in Plain English

Voyager is inspired by cartography: explorers are sent out to chart new territory, and a central command keeps track of which regions have already been covered.

The mechanics. The algorithm starts with a task prompt (for example, "Generate a poem") and maintains three things: a set of explorers (prompts derived from the task prompt), an anchor set of representative data points already generated, and the growing dataset itself.

Each iteration, an explorer prompts the LLM to produce a batch of instances. Every instance is scored by its marginal gain in volume — how much it would expand the geometric volume of the anchor set, measured through a similarity kernel. Instances above a threshold τ are accepted and added to the dataset and anchor set; instances below it are rejected. The anchor set is then pruned back down to a fixed size k by drawing a k-item sample from a DPP, which favors high-volume (i.e., internally diverse) subsets.

The feedback loop. The rejected instances are not discarded. They are shown to an LLM judge, which produces "textual gradients" — suggestions for how to change the prompt so future generations are more different from what already exists. A second LLM call applies those suggestions, producing new successor explorers. To keep things efficient, only up to b of these successors are kept, again via DPP sampling.

Why it works, mathematically. The authors show (Lemma 1) that the effective rank of the final similarity matrix — a proxy for diversity, as used by the Vendi Score — can be roughly approximated as V ≈ n²·D^(1/n)/C, where D is the determinant and C the trace. Maximizing the determinant therefore pushes effective rank up. Since directly maximizing the determinant of the whole dataset is infeasible, they reduce the goal to the maximum volume submatrix problem: find the size-k subset with the largest determinant. That problem is NP-hard, which is exactly why DPPs are used — a k-item DPP sample favors high-volume subsets and gives a rough lower bound on the maximum volume submatrix.

Cost. The reported total complexity is O(T(b(k_max³ + |B|k²) + b_max³)) across T iterations, compared to O(|l|³) for a naive approach that DPP-samples an oversized universe down to the target size. The expected number of outer-loop iterations is T_avg = l/(ζ|B|), where ζ is the fraction of candidates accepted in an average Explore call, giving an expected LLM-call count of N_llm = ρl/(ζ|B|) with ρ = 3 calls per Explore.

Experimental setup. The similarity kernel is a convex combination (0.7 RBF on text embeddings, 0.3 Jaccard lexical similarity), both of which are positive semi-definite. All data was generated with GPT-4o mini, and embeddings came from OpenAI's text-embedding-3-small. Voyager's hyperparameters were b = 3 explorers, k = 10 anchor points, T = 200 maximum iterations, |B| = 10 samples per LLM call, and target dataset size l = 500. The Hierarchical baseline generated 10 subtopics per call and 1 sample per subtopic. Quality was judged by a jury of GPT-4, GPT-4o, and GPT-4.1 using task-specific rubrics.

Why This Matters

Impact on research. The paper makes a case for treating diversity as a directly optimizable geometric quantity rather than a side effect of decoding tricks or topical prompting. Its connection between DPP volume, the maximum volume submatrix problem, and effective rank gives a reusable theoretical framing for anyone working on synthetic data quality. It also contributes an evaluation setup — cosine distance, Jaccard lexical distance, Vendi score, LLM-as-Judge quality, and LLM call count — that captures both the benefit and the cost of diversity methods.

Real-world applications:

• User persona modeling and personalization. The paper explicitly names this as a case where the benefit of diversity far exceeds the extra LLM-call cost.
• Synthetic training data for downstream models in domains where real-world data is scarce.
• Evaluation suite construction, where monotonous test sets under-measure model capability.
• Generating diverse creative or educational content — poems, movie plots, political conversations, grade-school math problems, and logic puzzles are all tasks tested here.

Industry relevance. Because the method is training-free and needs no access to weights or logits, it can be dropped on top of a closed-source API — the exact setting in which most production teams operate. The explicit LLM-call accounting (Voyager used 443, 426, 615, 695, 399, and 393 calls across the six tasks, versus 550 for Hierarchical on each) makes the cost/benefit tradeoff legible to a practitioner, and the authors state that the right point on that tradeoff is task- and application-specific. The Voyager-RE ablation gives practitioners a cheaper local-compute variant at a measurable diversity cost.

Future Directions

• Tuning the cost/diversity tradeoff. The paper leaves the choice between LLM calls and diversity gains as task-specific, and notes that Voyager-RE (random explorers) is a cheaper variant worth choosing in some settings. A principled way to pick between them remains open.
• Reducing reliance on the marginal-gain threshold τ. The anchor set is a fixed-size approximation of the whole dataset, and the paper acknowledges that computing marginal gain over the full, growing dataset would be prohibitively expensive. Better approximations may exist.
• Extending beyond the benchmarked task categories. Only creative writing (four tasks) and reasoning (two tasks) were tested, all with GPT-4o mini and one embedding model. Behavior on other modalities, other LLMs, or larger target dataset sizes is not reported.
• Reframing diversity metrics. The paper notes there is no universal consensus on how to measure diversity, which is why it reports three different metrics plus human evaluation. Whether one metric should dominate remains an open question.

Target Audience

This paper is most useful to machine learning engineers and applied researchers who build synthetic data pipelines on top of closed-source LLM APIs and need the resulting datasets to cover a wide semantic range. It is also relevant to researchers in data-centric AI and dataset curation who care about diversity measurement, and to practitioners who have hit mode collapse in large-scale generation jobs and want a method that comes with both theory and a cost model. Readers looking for a purely empirical benchmark paper will find the theoretical section demanding; readers looking for pure theory will find the empirical section is the main payload.

Authors’ abstract

Large language models (LLMs) are increasingly being used to generate synthetic datasets for the evaluation and training of downstream models. However, prior work has noted that such generated data lacks diversity. In this paper, we propose Voyager, a novel principled approach to generate diverse datasets. Our approach is iterative and directly optimizes a mathematical quantity that optimizes the diversity of the dataset using the machinery of determinantal point processes. Furthermore, our approach is training-free, applicable to closed-source models, and scalable. In addition to providing theoretical justification for the working of our method, we also demonstrate through comprehensive experiments that Voyager significantly outperforms popular baseline approaches by providing a 1.5-3 times improvement in diversity.