Emergency Response Measures for Catastrophic AI Risk

AI summary

Overview

Research area: AI safety and governance, specifically emergency preparedness policy for catastrophic AI risk, with a focus on China's regulatory framework.

Technical level: Intermediate. The paper contains no mathematics or model training details, but it references specific regulatory instruments, standards numbers, and benchmark names that assume some familiarity with AI policy.

Scope: A policy analysis arguing that frontier safety policies (FSPs) can operationalize the proactive phases of China's four-phase emergency response framework for catastrophic AI risk, comparing Chinese, EU, US state, and industry approaches.

What This Paper Is About

Chinese authorities have extended the country's four-phase emergency response framework (prevent, warn, respond, recover) to cover risks from advanced AI, but the concrete mechanisms for the proactive prevention and warning phases remain under development. This paper examines frontier safety policies — the pre-deployment dangerous capability evaluations and tiered, pre-planned safety measures used by leading AI companies — and argues they align closely with the proactive phases of China's framework. The goal is to show how the FSP model could operationalize AI emergency preparedness in a way consistent with China's established governance principles.

Key Contributions

1. A mapping of FSPs onto China's four-phase emergency response loop. The authors argue that FSP components — dangerous capability thresholds, pre-planned mitigations, and continuous evaluation — provide technical mechanisms for the first two phases (prevention and preparedness, surveillance and warning), which must be established before any emergency occurs.

2. A comparative survey of international AI emergency preparedness mechanisms. The paper reviews the EU Code of Practice for General-Purpose AI Models, California Senate Bill 53, the New York RAISE Act, the AI Seoul Summit Commitments, and company-level FSPs, identifying convergent risk categories and reporting timelines.

3. A concrete proposal for extending existing Chinese regulations. The authors describe how GB/T 45654-2025's documentation and assessment requirements could be augmented by mandating that providers maintain a frontier safety policy and by adding catastrophic-risk evaluations to the existing assessment suite.

4. An appendix cataloguing potential implementations for all four phases. Table 1 lists implementations such as information sharing among frontier developers, standardized dangerous-capability benchmarks, continuous query monitoring, hardware access restriction, blameless postmortems, and tabletop exercises.

Main Findings

• China's framework already provides the architecture. The four-phase system derives from the Emergency Response Law of the People's Republic of China and the "One Plan, Three Systems" framework that has been in place since the early 2000s. TC260 has already adopted a four-stage structure for AI emergency response.

• AI is classified alongside natural and national disasters. The National Emergency Response Plan released in February 2025 lists "artificial intelligence security" incidents next to earthquakes, cyberattacks, and infectious disease epidemics among emergencies requiring "mass monitoring and prevention." The paper defines catastrophic AI risk as risks of exceptionally destructive events that persistently affect many people over a wide geographic area, noting this could mean at least dozens of deaths or at least billions of dollars in damage.

• Threat models converge across jurisdictions. TC260's AI Safety Governance Framework identifies proliferation of weapons of mass destruction (biological, chemical, nuclear, cyber) and loss of human control as risks. The EU Code recognizes four catastrophic risk categories: CBRN attacks, loss of control, sophisticated cyber-attacks, and strategic manipulation of human behavior. OpenAI's Preparedness Framework sets thresholds for biological and chemical capabilities, cybersecurity capabilities, and AI self-improvement capabilities. California SB53 identifies CBRN weapons assistance, cyberattack capabilities, and scenarios where models might evade developer control.

• Incident reporting timelines are already specified in other jurisdictions. The EU Code details two days for incidents causing "serious and irreversible disruption of critical infrastructure" and five days for serious cybersecurity breaches. California SB53 mandates 24-hour reporting for incidents posing "imminent risk of death or serious physical injury." The New York RAISE Act mandates 72-hour reporting.

• China's existing regulations establish precedents but stop short of catastrophic risk. The Interim Measures for the Management of Generative AI Services (August 2023) require registration and pre-deployment security assessments under Article 17. The Emergency Response Guidelines for Security of Generative AI Services classifies incidents into ten categories drawn from GB/T 20986-2023 and defines four levels of incident severity. GB/T 45654-2025 (April 2025) specifies technical requirements across the model lifecycle and includes a thirty-one item list of security risks for pre-deployment assessment. The authors state the main limitation is that these focus on content security rather than catastrophic risks.

• Voluntary industry frameworks go further than regulation. The Frontier AI Risk Management Framework, released in July 2025 by the Shanghai AI Laboratory with the social enterprise Concordia AI, establishes a three-tiered risk classification with "yellow lines" as early warning indicators and "red lines" as unacceptable risk thresholds. It prescribes evaluations using benchmarks including WMDP-Bio and LAB-Bench for biological risks. Seventeen leading Chinese AI companies including Baidu, Alibaba, DeepSeek, and ByteDance signed the "Artificial Intelligence Safety Commitments" in December 2024 under the China AI Industry Alliance (AIIA); updated Commitments were issued at the World AI Conference in July 2025, with five additional companies signing.

• Twenty companies signed the AI Seoul Summit Commitments in May 2024, including Chinese companies MiniMax and 01.ai alongside OpenAI, Anthropic, Google, Microsoft, and Meta. Signatories commit not to develop or deploy a model at all if mitigations cannot keep risks below thresholds.

• Thresholds have already been crossed in practice. The paper notes that the first of Anthropic's Responsible Scaling Policy thresholds was surpassed in May 2025, leading Anthropic to enhance model weight security and make Claude Opus 4 more robust against jailbreaks.

• The paper reports no experiments, benchmarks, or quantitative results of its own. All figures and named evaluations are drawn from existing regulations, standards, and company policies.

Methodology in Plain English

This is a conceptual and comparative policy analysis rather than an empirical study. The authors first review China's existing regulatory instruments and voluntary industry initiatives, then survey emergency preparedness practices in the EU, at the US state level, and among AI companies internationally. They compare the risk categories, threshold structures, monitoring requirements, and reporting timelines across these sources to identify convergence. From that comparison they construct a proposal: extend China's existing generative AI registration and assessment requirements to include frontier safety policies and catastrophic-risk evaluations, and illustrate the change with a figure showing the registry-centric model lifecycle. A final section enumerates limitations and an appendix lists candidate implementations for each of the four emergency response phases.

Why This Matters

The paper attempts to bridge two governance traditions that are often discussed separately: China's institutionalized emergency management system and the FSP practices developed largely by Western AI labs. It argues that capability-based thresholds, tiered mitigations, and continuous evaluation can serve as real-time indicators of catastrophic risk, in the same way seismic monitoring supports earthquake early warning. It also identifies a competitive parity argument — that mandating FSPs would bring Chinese companies in line with safety standards already adopted by global competitors.

Real-world applications:

• Regulatory drafting. Regulators could add FSP documentation requirements and catastrophic-risk evaluations to existing pre-deployment assessment procedures, or include such obligations in future AI legislation.
• Standards development. TC260 or similar bodies could develop a standard operationalizing earlier draft language about self-replicating AI, malware writing, and biological or chemical weapons risks.
• Company compliance. AI developers could use the proposed structure to specify dangerous capability thresholds beyond current model capabilities and pre-plan corresponding mitigations.
• Incident response coordination. Notification channels between frontier model developers and government authorities, modeled on Anthropic's commitment to notify a relevant US Government entity for models requiring greater than ASL-2 security and Google DeepMind's commitment to share information with appropriate government authorities, could speed response during an emergency.

Industry relevance: the proposal would formalize practices that leading Chinese labs are already moving toward voluntarily, and it draws on mechanisms already operating internationally, including the Frontier Model Forum's risk and safety information sharing among OpenAI, Google, Anthropic, Meta, Microsoft, and Amazon, and Article 73 of the EU AI Act's rapid incident reporting mechanism.

Future Directions

• Extending FSPs to the response and recovery phases. The authors acknowledge FSPs focus primarily on prevention and monitoring, and that the response and recovery phases require additional implementations, which they sketch in Appendix A rather than fully develop.
• Defining China's own dangerous capability thresholds. The paper leaves open whether developers would set their own thresholds, in the spirit of SB53, or whether specific mitigations would be mandated at designated risk thresholds, much like NY RAISE would ban deployment above a risk threshold.
• Developing standardized benchmarks. A stated need is for more standardized dangerous capability benchmarks developed jointly by companies, government bodies, and civil society organizations to strengthen the evidentiary basis for both internal evaluation and external oversight.
• Turning regulatory standards into law. China's national legislature has yet to pass laws intended to regulate AI, though the State Council has repeatedly indicated such laws are being drafted. The paper points to the Model Artificial Intelligence Law proposed by the Chinese Academy of Social Sciences as a possible vehicle for requiring pre- and during-deployment risk assessment and pre-established "safety risk management systems."

Target Audience

Policymakers and regulators working on AI governance, particularly those involved in China's AI standards and emergency management systems; AI safety researchers studying governance mechanisms; compliance and policy staff at frontier AI companies; and students or analysts seeking a comparative overview of how China, the EU, US states, and industry have approached catastrophic AI risk preparedness.

Authors’ abstract

Chinese authorities are extending the country's four-phase emergency response framework (prevent, warn, respond, and recover) to address risks from advanced artificial intelligence (AI). Concrete mechanisms for the proactive prevention and warning phases, however, remain under development. This paper analyzes an implementation model inspired by international AI safety practices: frontier safety policies (FSPs). These policies feature pre-deployment evaluations for dangerous capabilities and tiered, pre-planned safety measures. We observe close alignment between FSPs and the proactive phases of China's emergency response framework, suggesting that the FSP model could help operationalize AI emergency preparedness in a manner consistent with China's established governance principles.