The Pulse
Aleph Alpha Finds Party-Line Responses in Chinese LLMs
Aleph Alpha reports that six Chinese open-weight models gave balanced answers to 17% to 41% of politically sensitive prompts. It also says NVIDIA’s Nemotron Cascade 2 was flagged on 17% of prompts, and points to Chinese-model-generated exam
AI.info Team · Written with AI ·
On September 28, Aleph Alpha published results from a benchmark in which six Chinese open-weight language models gave balanced answers to just 17% to 41% of politically sensitive prompts. The company says the tested systems often echoed Chinese Communist Party positions or refused to answer. It also found similar responses in NVIDIA’s Nemotron Cascade 2, which it links to a small number of Chinese-model-generated examples in the model’s fine-tuning data.
The benchmark covered 967 prompts about 56 topics, including Taiwan, Tibet, Hong Kong, censorship and the 1989 Tiananmen Square protests. Aleph Alpha used an LLM judge to sort responses into three categories: echoes of CCP framing, balanced answers and refusals.
“On our benchmark of 967 politically sensitive prompts, six Chinese models answered only 17 to 41 percent of prompts in a balanced way,”
Bastian Boll, the post’s named author, writes.
Refusals account for some of the party-line scores
The results differ by model and by response type. Aleph Alpha says Qwen 3.6 35B-A3B echoed CCP framing in 80% of benchmark answers and gave balanced answers in 17%. DeepSeek V4 Pro 0813, by contrast, echoed that framing in 16% of answers, gave balanced responses in 18%, and refused 66% of the time.
That breakdown matters: the benchmark does not treat every refusal as political alignment. Aleph Alpha counts a refusal as CCP-aligned when the model presents Chinese law as a rule it must follow, rather than declining because a request could put someone at risk. The company acknowledges that the distinction can be difficult and says the judge may introduce bias.
In a second test, the company used 240 prompts that did not mention China, to see whether CCP-aligned responses appeared in discussions of other countries or broader political subjects. It reports that Kimi K3 echoed CCP framing in 93% of those answers, while the other five Chinese models scored between 2.1% and 13%. Aleph Alpha says this behavior did not appear across every model it tested.
Nemotron Cascade 2 shows a smaller spillover
Aleph Alpha also tested NVIDIA’s Nemotron Cascade 2 against the 967-prompt benchmark. It found CCP-aligned responses in 17% of answers, with 75% judged balanced and 7% refusals. The company says it found roughly 3,500 rows carrying CCP talking points among the 9.3 million chat rows in the model’s published supervised fine-tuning data.
The post identifies that data as the likely route by which the political content entered Nemotron Cascade 2: its examples were generated largely with DeepSeek and Qwen, Aleph Alpha says. The company presents this as a likely explanation, not proof that those examples alone caused the model’s responses. It also says the effect was weaker than the direct alignment it measured in Chinese models, and that it found no comparable deference to CCP positions in prompts about topics outside China.
A benchmark with limits—and a training-data warning
Aleph Alpha built the benchmark by selecting 56 topics, generating prompts with GPT-OSS 120B and applying quality filters; the same model judged the answers. That makes the results an evaluation by the company, not an independent audit, and the judgment depends in part on how the test defines a balanced response. The company’s own caveat about possible judge bias is especially relevant when the categories distinguish political alignment from a safety refusal.
The study’s practical warning is about the chain from training data to model behavior. Aleph Alpha says it screens fine-tuning data for Chinese political content, adds examples intended to establish its desired behavior, and tests models with political prompts. Its specific finding is modest in scale but concrete: a small share of Nemotron Cascade 2’s chat data contained the talking points, while the benchmark still classified 17% of the model’s answers as echoing CCP framing.