ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation

AI summary

Overview

Research area: Natural Language Processing — inference-time generation strategies for large language models (multi-sample selection, reasoning, summarization, code generation).

Technical level: Intermediate. The paper combines a straightforward procedural pipeline (similarity graph, spectral clustering, centroid pick) with spectral graph theory and a theoretical argument linking degree centrality to kernel density estimation.

Scope in one sentence: The paper introduces ModeX and ModeX-Lite, evaluator-free Best-of-N selection methods that pick the "modal" output from multiple LLM generations by clustering them on a lexical similarity graph, and evaluates them on text summarization, code generation, and mathematical reasoning.

What This Paper Is About

When an LLM generates several candidate answers to an open-ended prompt, there is no obvious way to pick the best one: outputs often differ in wording while meaning the same thing, so exact string-match voting fails, and reward models or LLM judges are expensive and can be wrong. ModeX addresses this by treating selection as finding the "mode" of the generated set — building a similarity graph over candidate outputs, isolating the dominant cluster, and returning its most central member — with no external evaluator and no extra neural inference.

Key Contributions

1. ModeX, an evaluator-free Best-of-N selection framework that generalizes majority voting to open-ended text generation by identifying a modal output, without external evaluators or auxiliary scoring models.
2. ModeX-Lite, an efficiency-oriented variant that periodically applies modal selection and pruning during generation rather than only at the end.
3. Extensive experiments on three open-ended generation tasks (CNN/DailyMail summarization, HumanEval code generation, Math-500 mathematical reasoning), reported as state-of-the-art among evaluator-free approaches.
4. Theoretical justification: two theorems arguing that recursive spectral clustering isolates a modal region of the output distribution and that weighted degree within that cluster approximates a kernel density estimate, jointly constituting conditional mode estimation.

Main Findings

• Evaluator-free selection is competitive. ModeX and ModeX-Lite are reported to outperform the Single Path, Self-Refine, LLM Judge, Perplexity BoN, and Self-Certainty BoN baselines across the three benchmarks, and sometimes to surpass the gold-standard Best-of-16 that uses external reward models.
• Code generation with Qwen. The paper's text states that applying ModeX to Qwen with N=16 paths improves the mean Single Path baseline from 69.89% to 78.66% on Pass@1. In Table 1, the Qwen Pass@1 column lists 75.61 for ModeX (N=16) and 78.66 for ModeX-Lite (N=16).
• Summarization with Qwen. Single Path reports Rouge-1 32.95, Rouge-2 10.47, Rouge-L 20.17; ModeX-Lite (N=16) reports Rouge-1 35.78, Rouge-2 12.35, Rouge-L 21.89. The gold-standard Best-of-16 reports Rouge-1 33.46, Rouge-2 10.64, Rouge-L 20.49.
• Math reasoning with Llama. Single Path accuracy is 38.75; ModeX (N=16) reaches 49.33; the gold-standard Best-of-16 reaches 63.00. On the same task, LLM Judge improves only from 37.33 (N=4) to 38.67 (N=16).
• Scaling the number of paths helps ModeX more than judging. The paper reports that increasing N from 4 to 16 gives the LLM Judge baseline only a +1.34 percentage-point gain in Llama math accuracy, whereas ModeX-Lite gains +7.33 percentage points from the same increase.
• Extra self-refinement compute is not enough. Self-Refine consumes roughly 4x the compute of standard generation yet fails to surpass ModeX, and the paper reports it can drop significantly below the Single Path baseline (for example, Llama summarization Rouge-1 falls from 33.97 to 23.97).
• Efficiency. Measured per-sample wall time on CNN/DailyMail with Qwen-7B: Single Path 5.5s, Self-Refine 31.7s, LLM Judge 10.7s, Best-of-N 11.1s, ModeX-Lite (N=4) 7.2s, ModeX-Lite (N=16) 9.1s. The paper describes this as a 3.5x speedup of ModeX (N=16) over Self-Refine. Complexity for ModeX-Lite is O(NL + N²) versus O(L) for single-path.
• Quality is detectable early. On math reasoning with Qwen-7B, high-quality candidates are identifiable from partial generations even when less than 50% of the full trajectory has been produced — the motivation for pruning during generation.
• Hyperparameter robustness. ModeX-Lite performance stays above the single-path baseline across conductance thresholds τ in {0.5, ..., 0.8} (compared against a Normalized Cut alternative) and pruning intervals T in {100, ..., 500}.
• Qualitative behavior. In the CNN/DailyMail example, "rejected" summaries omit keywords, contain imprecise or erroneous details, or are repetitive and verbose, while "chosen" summaries are concise and focused.

Methodology in Plain English

ModeX treats each of the N sampled generations as a node in a graph.

1. Build the graph. Each pair of generations is connected with a weight equal to the sum of three Jaccard similarities computed over unigram, bigram, and trigram sets. Lexically similar outputs get stronger edges. This is the soft, open-ended analogue of the hard "clique of identical answers" used in majority voting.
2. Split the graph. The method uses the Fiedler vector (the second eigenvector of the graph Laplacian) to cut the graph into two parts, then measures the quality of the cut with conductance — the ratio of edge weight crossing the cut to the smaller side's total degree. Low conductance means a clean separation. The algorithm keeps the larger side (breaking ties by total edge weight) and recurses until no split achieves conductance below a threshold τ, set to 0.8 in the experiments.
3. Pick the centroid. From the surviving cluster, it returns the node with the highest total weighted degree, i.e. the response most similar to all the others. That response is treated as the approximate mode and returned as the final output.

ModeX-Lite applies the same idea but during generation: every T steps (T = 100 unless stated otherwise), it runs a single non-recursive spectral clustering over the partially generated trajectories and prunes the least representative ones, deferring the final centroid pick until generation finishes. Because transformer inference can generate the N paths in parallel, the added cost of ModeX-Lite's selection step, O(N²), is described as negligible relative to generation, O(NL).

The theoretical section argues two things: that as the number of samples grows, the spectral bipartition converges to the cut separating distinct semantic manifolds, and that weighted degree in a cluster is proportional to a kernel density estimate of the underlying distribution — so clustering followed by centroid selection amounts to estimating the mode of the dominant component.

Why This Matters

Impact on research. The paper argues that gains from multi-path generation depend on structured aggregation rather than on simply sampling more, and it frames ModeX as a formal generalization of majority voting to open-ended generation. It provides an evaluator-free alternative to reward-model-based Best-of-N and to LLM-judge selection, and offers the first step toward selection methods that use only the relational structure of the sampled texts.

Real-world applications (as targeted by the benchmarks):

• Summarization pipelines where many candidate summaries can be generated in parallel and a concise, keyword-covering one must be chosen without a human or reward model in the loop.
• Coding assistants that sample several completions and pick one, with Pass@1 measured on HumanEval.
• Mathematical reasoning or tutoring assistants that need to filter unreliable reasoning traces, measured on Math-500.
• Deployments where an external reward model or judge LLM is unavailable, too expensive, or undesirable due to latency and privacy constraints.

Industry relevance. Selection quality here comes at a latency cost close to single-path generation (7.2s for N=4 versus 5.5s for single path on the reported setup) rather than the 31.7s reported for iterative self-refinement, which matters for latency-sensitive serving. Avoiding a reward model also removes a dependency that the paper notes does not exist for code generation, where no good reward model was available for the Best-of-N gold standard. The paper also acknowledges that multi-path generation increases aggregate energy consumption per query.

Future Directions

• Better similarity measures. The method currently relies on lexical Jaccard similarity over n-grams; the authors suggest embedding-based or otherwise richer similarity measures that can recognize valid paraphrases with different surface forms.
• Guarding against consensus on error. If the model systematically "mode collapses" toward a specific hallucination, ModeX may identify and reinforce that erroneous consensus. The authors call for work to mitigate these corner cases.
• Adaptive generation–selection strategies. The paper points to adaptive schemes for deciding when to prune or select as a route to further robustness and efficiency.
• Better reward models for code. Since no good reward model was available for HumanEval, the Best-of-N gold standard for code generation is left unreported (shown as "–"), leaving that comparison open.

Target Audience

Researchers and engineers working on LLM inference-time scaling, decoding, and multi-sample selection benefit most, along with practitioners who need reliable best-of-N selection without training or deploying reward models or judge LLMs. It is also relevant to readers interested in applying spectral graph methods to natural language outputs, and to those evaluating the cost–quality tradeoff of parallel generation versus iterative self-refinement.

Authors’ abstract

Selecting a single high-quality output from multiple stochastic generations remains a fundamental challenge for large language models (LLMs), particularly in open-ended tasks where no canonical answer exists. While Best-of-N and self-consistency methods show that aggregating multiple generations can improve performance, existing approaches typically rely on external evaluators, reward models, or exact string-match voting, limiting their applicability and efficiency. We propose Mode Extraction (ModeX), an evaluator-free Best-of-N selection framework that generalizes majority voting to open-ended text generation by identifying the modal output representing the dominant semantic consensus among generated texts. ModeX constructs a similarity graph over candidate generations and recursively applies spectral clustering to select a representative centroid, without requiring additional inference or auxiliary models. We further instantiate this selection principle as ModeX-Lite, an improved version of ModeX with early pruning for efficiency. Across open-ended tasks -- including text summarization, code generation, and mathematical reasoning -- our approaches consistently outperform standard single- and multi-path baselines, providing a computationally efficient solution for robust open-ended text generation. Code is released in https://github.com/deeplearning-wisc/ModeX.