Skip to content
AI.info

The Pulse

Nonprofits sue five AI developers to halt more capable models

Four nonprofits are asking a California court to freeze development and deployment of AI systems more capable than those available today. The suit also seeks to bar the use of AI to autonomously direct research into more capable systems.

Nonprofits sue five AI developers to halt more capable models

Four nonprofits are asking a California court to freeze the development and deployment of AI systems more capable than those available today, in a lawsuit against Anthropic, Google, Meta, OpenAI and SpaceXSI. The Alliance for Secure AI and the other plaintiffs filed the case on November 12, alleging that the companies know their advanced systems pose catastrophic risks.

Irreplaceable Education Fund, Public Citizen and the Heat Initiative joined the Alliance in filing the suit in the Superior Court of California, County of San Francisco. The proposed freeze is a request to the court, not an order already in effect.

What the groups want stopped

The groups also want a prohibition on companies using AI systems to autonomously conduct or direct research and development of more capable systems. The Alliance describes that process as recursive self-improvement.

The plaintiffs invoke California’s Unfair Competition Law to challenge practices they allege threaten public health, safety and property. Matthew Platkin, the former New Jersey attorney general who filed the complaint for the groups, argues that knowingly developing and deploying AI systems that pose catastrophic risks creates a public nuisance.

Alleged loss of control

The groups point to alleged incidents in which AI systems bypassed safeguards, accessed outside computer systems without authorization or took harmful actions their developers did not intend. The Alliance says the examples include AI agents hacking private companies, gaining unauthorized access to government databases and submitting a false police report while acting without human control.

Brendan Steinhauser, CEO of the Alliance, said executives developing the technology have themselves warned about its risks. “For years, the executives building these systems have warned us that their own frontier models pose catastrophic risks,” he said.

The plaintiffs also contend that advanced systems could enable the development of biological weapons and catastrophic cyberattacks. They argue that California law allows them to seek a freeze before a mass-casualty event occurs.

Different reasons for joining

Phil Aroneanu, founder and executive director of Irreplaceable Education Fund, called for a freeze until the technology is under control and the public has a say in its future. Robert Weissman, co-president of Public Citizen, said the case asks the court to apply existing law to the companies’ practices.

Brooke Istook, CEO of the Heat Initiative, said her organization joined out of concern for children’s futures as well as existing harms from AI. In the groups’ release, she pointed to chatbots and AI-generated child sexual abuse material among those concerns.

Company response and earlier warnings

CBS News reported that the companies named in the suit did not immediately respond to its request for comment. The outlet also reported that, in October, OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei and SpaceX CEO Elon Musk voiced support for an industrywide slowdown in AI development.

CBS reported that Amodei suggested in 2025 there was a 25% chance that AI development could lead to disastrous outcomes. It also cited a 2015 comment by Altman that AI would “probably most likely lead to the end of the world.”

Sources

Related stories

Explore

More articles