The Pulse
Two Opening Tokens Bring Base Models Near RL on Math Tests
A paper reports that prefilling selected response openings substantially improved math benchmark scores for two base models, bringing them close to reinforcement-learning-trained counterparts. The researchers also found that changes to trai
AI.info Team · Written with AI ·
A period-and-blank-line opening followed by “Okay” lifts Olmo-3-7B’s score on MATH-500 from 42% to 78%, nearly matching the 75% score of its reinforcement-learning-trained counterpart. The result appears in a paper submitted to arXiv on October 5, 2026, by Sophie L. Wang, Amil Dravid, Rulin Shao, Kevin Farhat, Sewon Min and Alexei A. Efros. Their experiments suggest that a model’s first few response tokens can steer reasoning behavior it has already learned.
Olmo-3-7B gains 36 points on MATH-500
The researchers call the openings “token cues.” Rather than changing a model’s weights, they prefill the start of its response and let it continue. For Olmo-3-7B, the selected cue is a period, a blank line and “Okay”; for Qwen3-14B, it is “Alright,”.
On MATH-500, the Qwen3-14B cue raises pass@1 accuracy—the chance that one sampled response is correct—from 72% to 87%, matching the study’s RL-trained Qwen counterpart. The authors also tested GSM8K, AMC 2023, AIME 2024 and HumanEval, reporting improvements on several math and coding comparisons. They say the cues often match or exceed the RL results in those tests, though Qwen’s HumanEval score stays close to its already-high uncued baseline.
“In this paper, we study how training data creates associations between the tokens at the start of a base model’s response and the reasoning behavior that follows.”
The authors write: Sophie L. Wang, Amil Dravid, Rulin Shao, Kevin Farhat, Sewon Min and Alexei A. Efros, paper authors
The cue changes the opening, not the model’s weights
The comparison is specific to the models and setups in the paper. The team evaluated Olmo-3-7B and Qwen3-14B against RL-trained counterparts, sampling 32 responses per problem and using a response budget of up to 31,744 tokens unless stated otherwise. The paper defines pass@1 as the probability that a single sampled response is correct.
The researchers searched for effective openings without using answer labels: they proposed likely starts and selected among them based on agreement across sampled answers. They also tested whether the cue’s benefit was explained just by longer outputs. In a comparison of 20 candidate openings on MATH-500, using four responses per problem and a 31,744-token budget, Olmo’s average response length increased from 4,800 to 6,700 tokens with the selected cue. The paper also reports that the cue beat the uncued model when responses were compared at matched token limits.
RL makes the same openings more likely
The paper traces part of RL’s measured gain to the response opening. It reports that the largest differences between the base and RL models’ next-token distributions appeared in the first two output positions. Figure 3 shows probabilities for individual tokens conditioned on the prompt and the preceding branch—not probabilities for producing each complete cue—so the results indicate that RL shifts probability toward tokens in the selected openings.
In a further test, the researchers prefixed the cue before RL training updates. On their reported learning curves, that move matched about 100 RL training steps for Olmo and 50 for Qwen; subsequent cue-forced training added little before standard RL caught up. The experiment does not show that the opening accounts for every effect of post-training, but it indicates that a small change at the start can recover much of the measured accuracy gain in these runs.
Changing the training data changes the cue
To test whether the effect came from learned associations, the team edited Olmo’s mid-training data and retrained from an intermediate checkpoint. In this experiment, they trained on a 10-billion-token mixture, replacing whole-word uses of “Okay” with “Chicken,” then evaluated on MATH-500 under the RL-Zero prompt with a 7,168-token response budget and 32 responses per problem. With the period-and-blank-line “Chicken” cue, accuracy rose from 2.4% on the original-data model to 37.2% on the edited-data model. The researchers also changed training examples so the instruction “Think duck duck goose” worked about as well as “Think step by step” at eliciting reasoning.
Those results link the cue effect to the text used in training, rather than to a special reasoning property of the words themselves. The authors also report that different openings can shift refusal and compliance behavior. In Olmo-3-7B, “Okay,” reduced refusals and increased harmful responses to unsafe requests, while the period-and-blank-line “Okay” cue produced more selective refusal behavior.
Strong benchmark results, narrow evidence
The findings offer a focused account of how a base model can produce different behavior from the same weights: its opening tokens steer which learned patterns shape the continuation. That matters for interpreting reasoning benchmarks, because a gain attributed to RL may partly reflect a changed tendency to begin responses with openings associated with deliberation.
The evidence is not a demonstration that a short prefix can replace RL across models, tasks or real-world use. The comparisons cover selected models, prompts and benchmarks, and the safety tests show that cue choice can affect refusals as well as problem solving. In the paper’s MATH-500 comparison, the period-and-blank-line “Okay” opening takes Olmo-3-7B from 42% pass@1 to 78%, against 75% for its RL counterpart.