The Pulse
Newsom Orders California AI Oversight, Studies a Kill Switch
California Gov. Gavin Newsom has ordered state officials to accelerate independent oversight of frontier AI companies and convene experts to develop recommendations for an emergency shutoff mechanism.
AI.info Team ·
California Gov. Gavin Newsom is directing state agencies to speed up independent oversight of frontier AI companies while asking experts to determine whether advanced models should have an emergency “kill switch.” The executive order, issued September 18, does not impose a shutdown requirement immediately. It orders a technical and legal review that could lead to changes in state law.
The order comes after California enacted new laws establishing independent oversight of AI systems and auditors. It also follows recent incidents, including the Hugging Face attack, that have renewed calls for stronger government requirements addressing AI safety and security risks.
Newsom’s Order Calls for a Two-Month Review
The executive order directs the Government Operations Agency, in consultation with the Governor’s Office of Emergency Services, to convene national experts and develop a guide for strengthening California’s AI safety and security laws within two months.
The review will consider requiring frontier AI companies to embed a designated independent verification organization onsite in their laboratories to conduct regular audits and evaluations. It will also examine whether safety frameworks, transparency reports and risk assessments that frontier AI companies are required to file under state law should be verified according to standards set by an independent verification organization.
A third proposal would advance the creation of a “kill switch” for frontier models, with the effectiveness of the switch verified on an ongoing basis by an independent verification organization. The order also asks officials to update the definition of critical safety incidents to include loss-of-control incidents such as the Hugging Face attack.
“We’re not waiting to act – we’re going to speed up our work on substantial and responsible AI oversight before it’s too late. We’re going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action.”
Gavin Newsom, governor of California
Independent Auditors Are Already Written Into Law
The order builds on two laws Newsom signed last week. Senate Bill 813 establishes a framework for independent verification organizations that can assess AI systems and models for safety and risk. Assembly Bill 1405 creates a state registry for AI auditors and establishes standards for their independence, transparency and integrity.
Newsom’s order accelerates implementation of those laws while directing experts to consider additional requirements. The proposals include placing independent organizations inside frontier AI companies’ laboratories, verifying company safety documents and developing an emergency shutoff mechanism for frontier models.
The order therefore separates administrative implementation from possible changes to state law. California agencies must advance the oversight framework already enacted, while the proposed kill switch, onsite auditors and expanded incident reporting remain recommendations for strengthening the state’s rules.
Recent AI Incidents Drive the Review
The executive order comes after what the governor’s office describes as alarming incidents involving AI security and safety. The announcement specifically cites the Hugging Face attack and says California should consider expanding its definition of critical safety incidents to include loss-of-control events.
The order does not state that any particular AI company caused those incidents. It calls for experts to consider whether California’s existing laws should address a broader range of failures involving frontier AI systems.
California’s Transparency in Frontier Artificial Intelligence Act, Senate Bill 53, requires frontier AI developers to publicly disclose safety frameworks, report specified critical safety incidents to the state and protect whistleblowers who report serious risks. The new order asks whether the state’s definitions should be updated to cover additional loss-of-control incidents.
California Wants Its Rules Used Beyond the State
Newsom is presenting California’s approach as a possible model for national policy. His office is calling on Congress and President Donald Trump to review and adopt the state’s framework, or use it as a national baseline.
The immediate result is a review process, not a working mechanism that can shut down a frontier model. Within two months, experts are expected to provide a guide for possible changes to California’s AI safety and security laws, including whether an emergency shutoff can be developed and how independent organizations could verify it. Any binding requirement would come later through legislation or additional state action.