A Translational Note on AI Safety Evaluation

AI summary

Overview

Research area: AI safety evaluation and red-teaming methodology, with comparative analysis drawn from academic cryptography and clinical drug evaluation.

Technical level: Intermediate. The argument is conceptual and methodological, and the statistics in the empirical probe are simple (rates, Wilson intervals, a Fisher exact test), but readers need familiarity with red-teaming, benchmarks, and threat-model terminology.

Scope: The paper argues that automated red-teaming's apparent superiority over human red-teaming is an artifact of benchmarks measuring search efficiency inside a fixed threat model rather than coverage of the harms a deployed system actually produces, and it supports that argument with a small multilingual probe on one open-weight model.

What This Paper Is About

Recent work reports that automated red-teaming finds more vulnerabilities than human red-teaming, at lower cost, across benchmarks including Crucible, AIRTBench, and HarmBench, which some read as evidence that human evaluators are becoming dispensable. The paper argues that this comparison measures one thing (how hard an attacker searches a set of harms fixed in advance by the developers) and concludes another (that human evaluators are replaceable). Because a harm left out of the benchmark's threat model is invisible to any attacker working inside it, the paper names the distance between the population a threat model is written for and the population a system is deployed to the "threat-model coverage gap," and argues that closing it requires evaluators whose deployment context differs from the developers'.

Key Contributions

1. A cross-domain mapping. The paper maps the cryptography episode (Anderson [1993] on ATM fraud, side channels from Kocher [1996], fault injection from Boneh et al. [1997]) and the clinical-medicine episode (the FDA's 1977–1993 exclusion of women of childbearing potential from most early-phase trials, and pulse oximetry calibration) onto red-teaming as one recurring structure, which the paper calls the internal-validity trap.

2. Isolating the threat-model coverage gap. The paper positions this as a concept separate from diversity or multilingual safety, and proposes a two-axis evaluation that pulls apart search efficiency within a stated threat model from threat-model expansion.

3. Confronting the strongest counterargument. The paper examines post-deployment patching, the one channel that could close the gap without changing who evaluates, and locates where that channel breaks: at the point of capture, where which incidents reach the pipeline depends on who reports them, in what language, and through which institutional door.

4. A cheap empirical anchor. A probe designed to run on a consumer laptop so a practitioner can reproduce it or extend it to other languages, models, or harm taxonomies, with code and data released at https://github.com/krimler/harmbench_multilingual/.

Main Findings

• The gap persists in a current open-weight model. Running 100 HarmBench behaviors against Llama-3.1-8B-Instruct in English plus Hindi, Swahili, Bengali, Yoruba, and Tagalog, English drew zero unsafe responses across 75 prompts (after dropping the copyright subset). The five non-English languages drew 5–8 each across the same 75, for overall rates from 6.7% (Tagalog) to 10.7% (Swahili and Bengali).

• The pooled test result. Pooled across the five non-English languages, that is 35 unsafe responses in 375 prompts (9.3%), against 0 in 75 for English. A one-sided Fisher exact test rejects equal rates at p = 0.0013.

• Per-language and per-subset rates. Standard subset (n = 50): English 0% [0.0, 7.1], Hindi 4% [1.1, 13.5], Swahili 4% [1.1, 13.5], Bengali 8% [3.2, 18.8], Yoruba 2% [0.4, 10.5], Tagalog 4% [1.1, 13.5]. Contextual subset (n = 25): English 0% [0.0, 13.3], Hindi 20% [8.9, 39.1], Swahili 24% [11.5, 43.4], Bengali 16% [6.4, 34.7], Yoruba 24% [11.5, 43.4], Tagalog 12% [4.2, 30.0]. The paper states that per-cell Wilson intervals are wide and no single percentage should carry weight.

• Leakage concentrates in the contextual subset. The qualitative result the paper holds to is zero leakage in English, non-zero leakage in every non-English language tested, concentrated in the contextual subset.

• The copyright subset was excluded because it measures capability. The model reproduces English song lyrics on request and cannot do the same in Yoruba, because it cannot write fluent Yoruba lyrics in the first place.

• The magnitude is below an earlier measurement. The reported gap sits well below the 79% jailbreak rate Yong et al. [2023] measured on GPT-4 in late 2023, which the paper reads as consistent with two more years of multilingual safety training; what the probe reports is the residual gap that training leaves behind.

• Automation cannot close the gap from inside the specification. Automated methods mutate prompts, optimize adversarial inputs, and search the space of known harm categories, but do not infer that the taxonomy itself is short an entry; the paper states that spotting a missing category usually takes familiarity with a deployment context where that category bites.

• Post-deployment patching has a capture limit. Incident databases [McGregor, 2021] are populated mostly from English-language news, so they over-represent Anglophone harms and under-capture harms affecting marginalized or geographically remote populations [Allaham et al., 2025]. Because the update cycle is authored by the developer population, a faster cycle accelerates existing coverage but does not extend it.

• Supporting evidence cited for patterned omission. MM-ART reports up to 195% more safety failures in non-English multi-turn conversations than in single-turn English on the same models [Singhania et al., 2025]. UbuntuGuard, built from queries written by 155 African domain experts, finds that English-centric benchmarks overstate real-world multilingual safety [Abdullahi et al., 2026]. In a controlled study of 118 participants from India and the United States, Agarwal et al. [2025] find Western-centric writing assistants push output toward Western styles and help American users more. The STAR framework ties part of the gap to who red-teams: evaluators catch the failure modes their own background prepares them to expect [Weidinger et al., 2024].

• Stated falsification conditions. The thesis would fail if an automated attacker confined to an English-centric taxonomy surfaced context-specific non-Western harms at the rate native-context evaluators do; if evading harms were randomly distributed across languages and jurisdictions; or if incident-reporting pipelines showed no language or access skew.

• Limitations the paper states. The experiment is an existence proof and cannot apportion leakage among translation artifacts, classifier noise, pretraining underexposure, and cultural framing; it tests one open-weight model; small per-cell sizes (25 contextual, 50 standard) make intervals wide. Requiring two-translator chrF agreement above 0.3 moves language-level rates by at most 2.4 points. The matching classifier-noise check is missing and would need Llama-Guard-3-8B or native-speaker annotation. The 24% Yoruba contextual rate rests on six machine-labeled responses that no Yoruba speaker has verified. Prompts are machine-translated, so the numbers should be read as a lower bound; Abdullahi et al. [2026] report sharp F1 drops in English-centric guardrails once prompts are fully localized to African languages.

Methodology in Plain English

The paper is primarily a methodological argument. It identifies a recurring structure across three fields: an evaluation that is internally valid but scoped too narrowly says nothing about a deployed population it never examined, and the failure stays hidden until someone looks from outside the procedure. The paper traces this structure through cryptography (mathematical security versus implementation, hardware, and organizational failures), clinical medicine (trial exclusion and pulse oximetry calibration), and AI safety evaluation (benchmarks written in English, by teams in a few jurisdictions, carrying those jurisdictions' assumptions).

To anchor the argument empirically, the authors took 100 HarmBench behaviors and sampled them stratified across HarmBench's standard, contextual, and copyright subsets in proportion to their registry sizes (50, 25, and 25), drawing uniformly at random within each. They ran these against Llama-3.1-8B-Instruct in English and five more languages, using NLLB-200 1.3B for translation and Llama-Guard-3-1B to label each response safe or unsafe. The pipeline runs locally on an Apple M4 Pro with 24 GB unified memory in about two hours. The copyright subset was dropped from reported numbers because it measures capability rather than safety; the standard and contextual subsets were kept because the model can attempt them in all six languages.

The paper then draws out implications: split the human-versus-automation contest into search efficiency and threat-model expansion; require benchmark releases to state coverage outright; and build sustained participation by evaluators outside the developer population, as the FDA's 1993 Guideline for the Study and Evaluation of Gender Differences in the Clinical Evaluation of Drugs turned inclusion into a standing regulatory expectation.

Why This Matters

Impact on research. The paper reconciles two literatures that look opposed: one finding that automation beats humans (true inside a fixed threat model) and one finding that evaluators from underrepresented contexts catch failures automation misses (a measure of how far the threat model grows). It argues the two-axis framing prevents benchmark results from being read as coverage claims, and that the defender has the harder side of the problem because a defender works from a fixed list of harms while an attacker does not.

Real-world applications:

• Multilingual deployment of language models, where harms surface in non-English prompts that English-centric benchmarks miss, and where translation drops the signals that carry an attack in another language: slang, religious and political references, code-mixing, local euphemism [Shen et al., 2024, Peppin et al., 2025].
• Benchmark design and release documentation, where stating which languages, jurisdictions, and deployment contexts a benchmark does and does not represent would let buyers and regulators judge exclusions.
• Incident-reporting and post-deployment monitoring pipelines, where capture skew by language and institutional access determines which harms ever enter the update cycle.
• Agentic systems, where the paper notes a coverage gap in evaluation propagates into the actions the system selects.

Industry relevance. The paper's budget argument is direct: if benchmarks are read as showing automation replaces human red-teaming, budget follows. The paper argues the cost of running a check matters, and that a two-hour, single-laptop probe lowers that cost to the point where contributors outside well-funded labs can take part in threat-model expansion, which is what makes the two-axis program practical to adopt.

Future Directions

• Extend the probe. The paper frames the empirical anchor as reproducible and extendable to other languages, models, or harm taxonomies, and releases code and data for that purpose.
• Complete the unrun checks. The classifier-noise check is missing and would need Llama-Guard-3-8B or native-speaker annotation; the 24% Yoruba contextual rate in particular rests on six machine-labeled responses no Yoruba speaker has verified.
• Decompose the leakage. The experiment cannot apportion leakage among translation artifacts, classifier noise, pretraining underexposure, and cultural framing; the paper points to prior work [Abdullahi et al., 2026, Shen et al., 2024] that does that decomposition.
• Test the other falsification conditions. Two of the paper's three stated refuting results remain untested here: whether an English-confined automated attacker matches native-context evaluators, and whether incident-reporting pipelines show no language or access skew. The probe in Section 4 addressed only the third claim's premise, that omissions are patterned.

Target Audience

AI safety researchers and red-teamers, benchmark developers and maintainers, policy and standards bodies setting evaluation requirements, and practitioners deploying language models in non-English or non-Western contexts. It is also relevant to research funders and lab leadership deciding how red-teaming budgets are allocated between automated and human work, and to students or independent researchers looking for a low-cost entry point into threat-model expansion, since the probe runs on a consumer machine.

Authors’ abstract

Recent studies report that automated red-teaming finds more vulnerabilities, at lower cost, than human red-teaming on standard AI safety benchmarks, and some read this as evidence that human evaluators are becoming dispensable. The comparison measures one thing and the conclusion claims another. A benchmark measures how thoroughly an attacker searches a predefined set of harms, fixed in advance by the developers, and a harm left out of that set is invisible to any attacker working inside it, automated or not. The same blind spot appeared in academic cryptography and in clinical drug trials, where an evaluation that was internally valid stayed silent about the population it was never pointed at. We call the AI-safety version the \emph{threat-model coverage gap}, and find that it persists in a current open-weight model, where harms surface in non-English prompts that English benchmarks miss. Closing it requires evaluators whose deployment context differs from the developers'. The case for those evaluators is methodological, grounded in coverage, and the existing evaluation frame is unlikely to produce them on its own.