The Pulse
Turn-Level Best-of-N Sampling Can Improve LLM Agent Accuracy and Consistency on Long-Horizon Tasks
A preprint evaluates a method that generates several candidate actions at each turn and uses a language model to select one for execution. The authors report accuracy gains of 5–10%, improved consistency across attempts and changes in agent
AI.info Team · Written with AI · fact-checked ·
Turn-level best-of-N sampling can lead to accuracy improvements of 5–10% and greater consistency across repeated attempts on complex, long-horizon terminal tasks, according to a preprint submitted to arXiv on 4 November 2026. The method, called Turn-BoN, uses a generator model to produce candidate actions at each turn and a selector model to choose one for execution in the environment.
Turn-BoN selects an action before it is executed
The paper evaluates Turn-BoN as an inference-time scaling approach for large language model agents. At each turn, a generator proposes N candidate actions, and a selector chooses one. Only the selected action is executed in the environment; the other candidates do not require sandbox execution.
The authors investigate whether off-the-shelf, open-weight language models can compare candidate actions directly, without training a separate action-value critic. They describe the approach as related to prior work using critic models for guided search at inference time, while testing direct comparison of candidate actions.
The paper distinguishes Turn-BoN from trajectory-level best-of-N sampling. In trajectory-level sampling, multiple complete agent trajectories are generated in parallel and then scored using an outcome reward model or an LLM judge. Because selection takes place after the trajectories are complete, it cannot change the intermediate actions taken during those attempts.
Turn-BoN instead uses additional computation at each turn to choose among proposed actions before one is carried out. The authors note that other search methods, including depth-first search and Monte Carlo tree search, explore alternative actions from intermediate states. Applying those methods to interactive agents can require checkpointing and restoring earlier environment states, which may be impractical in some settings.
The study reports gains and changes in agent behaviour
Experiments on long-horizon terminal tasks show improvements in accuracy and consistency across repeated attempts. The paper also reports improvements in pass rates, including pass@1 and pass@k, across the evaluated generator models, selector models and sampling widths.
The authors analyze agent trajectories to examine how the method changes behaviour over complete attempts. They report fewer executed turns and fewer undesirable behaviours related to looping, testing and debugging, including in unsuccessful attempts. Their abstract also identifies improved task instruction following, debugging and testing among the observed behavioural patterns.
The paper frames Turn-BoN as approximate, sampled one-step policy improvement at inference time: a selector chooses from actions proposed by the generator, with the aim of improving on the generator’s rollout policy. The authors derive conditions based on the expected advantage of selected actions under which the induced policy improves on that policy.
The approach adds inference-time compute
Turn-BoN requires generating multiple candidate actions and invoking a selector at every turn, so it incurs substantial additional inference-time compute. The paper identifies the development of more computationally efficient approximations that retain the method’s performance gains as an open question.
The preprint is titled “On Turn-Level Best-of-N Sampling for Scaling LLM Agents.” Its authors are Jalaj Bhandari, Dhawal Gupta, Hongli Zhan, Zhenting Wang, Varad Pimpalkhute, Zhengzhong Liu, Eric Xing and Rupesh Srivastava.