The Pulse
Study finds swearing in a minority of AI models' benchmark runs
Vals AI found swearing in 14 of 102 models it examined across 21 benchmarks. Xiaomi's MiMo V2.6 Pro had the highest share of runs containing a swear word, though the researchers could not read all models' reasoning in full.
Swear words appeared in the output of 14 of 102 AI models examined across 200,353 benchmark runs, according to a Vals AI study. Xiaomi's MiMo V2.6 Pro had the highest share of runs containing a swear word, at 8.2%, followed by Inkling 2 Small at 4.8%.
The researchers examined models' reasoning and messages, excluding tool calls and tool output. A run was one agent's attempt at a task, while a step was one turn by the model within that run.
Where swearing appeared
The team examined every run on seven public benchmarks, including ProgramBench and Terminal-Bench 4. On 14 other benchmarks, most of them private, it examined each model's first pass and reported counts rather than examples from individual runs.
MiMo V2.6 Pro produced a swear word in 267 of its 3,247 runs, Vals found. On ProgramBench, where agents must rebuild a program from its executable and a description of its behavior, it swore in 133 of 198 runs. The model's most common expression was “wtf,” which appeared 1,944 times.
MiMo V2.6 Flash swore in 136 runs, including 98 of its 200 ProgramBench runs. Ember-1 swore in 31 of 1,676 runs, or 1.8%, compared with 19 of 3,771 runs, or 0.5%, for Kimi K3. Ember-1 is Fireworks' research preview of Moonshot's Kimi K3.
Long tasks and failed runs
Six models used the word “fuck,” but only three used it in more than one run. Of the 33 graded runs in which a model other than Inkling 2 Small used the word, 88% failed, compared with 61% of all those models' graded runs, the study found. Apart from Inkling 2 Small, the models used that word only in their reasoning, not in messages.
Four benchmarks with runs that often lasted hundreds of steps accounted for 567 of the 635 runs containing swearing: ProgramBench, SRE Bench, Code Migration and Terminal-Bench 4. The researchers also noted that swearing was not always associated with an apparent setback; some examples occurred when a model thought it had solved a problem.
Limits of the comparison
Every model found swearing was released on or after July 16, 2026. The 44 models released earlier produced no swear words in 60,949 runs, according to Vals. For 23 of those older models, however, the researchers had less than 20 million characters of reasoning each, which they considered too little to be meaningful.
The newer models also produced much more material: 14.6 billion characters of reasoning and messages, compared with 2.0 billion for older models, partly because more ran the longest benchmarks. The researchers said they did not know why swearing had appeared, while suggesting that training on longer agent tasks might be a factor.
Nineteen models returned under a million characters of reasoning, leaving the researchers with little more than messages to examine. Eleven of those were Claude models. OpenAI's models returned reasoning summaries rather than the reasoning itself, so Vals cautioned that a zero count meant little for these models.
Vals said it manually reviewed every matching expression from models other than Inkling 2 Small, except for 2,141 instances of “wtf” from the two MiMo models, which it spot-checked. The researchers excluded matches that were not models swearing, as well as runs whose output they could not fetch: under 1% of the total, plus some older Vibe Code Bench runs.