The Pulse
NYC Council Proposes AI Audits, Kill Switches and Whistleblower Rewards
New York City Council Speaker Julie Menin announced bills requiring third-party validation and human shutdown controls for AI systems marketed or deployed in the city. The package also proposes whistleblower rewards, new ways to sue over ce
AI.info Team ·
New York City Council Speaker Julie Menin says the city can support AI innovation while imposing new safety rules; her legislative package would make independent checks and a human shutdown control conditions for marketing or deploying AI systems in the city. The proposals, announced September 25, also include a share of recovered penalties for whistleblowers and a route for some people harmed by jailbroken AI tools to sue. They are proposals, not enacted laws, and the Council plans to examine them at a hearing on October 5.
Outside validation—and a switch to stop the system
The broadest measure, Introduction 2602, would prohibit businesses from marketing, offering for sale or deploying an AI system in New York City without third-party validation. Validators would check data quality, bias, decision outputs, privacy and security, along with any additional requirements set by the city’s Office of Cyber Command. They would also have to disclose conflicts of interest related to the system.
The same proposal requires a human override capable of shutting down each covered system, with the validator responsible for confirming the switch exists. Businesses and validators could each face a $25,000 penalty for every instance of a system marketed, offered for sale or deployed without validation, or where validation was falsified. The proposal places responsibility on both the company using or selling a system and the outside party checking it.
Whistleblowers could share recovered penalties
Introduction 2605 would let individual whistleblowers receive a portion of fines or penalties recovered from AI companies that violate applicable laws. The Council describes the proposal as a first-in-the-nation incentive, but its announcement does not specify what share a whistleblower could receive. A separate bill, Introduction 2604, would clarify that city employees and covered contractors and subcontractors are protected when they report AI-related conduct they know or reasonably believe poses a public-safety threat.
Another proposal would give people a private right of action for foreseeable harm caused when a third party circumvents an AI company’s safety controls. Under Introduction 2600, a claimant would need to show that the harm was foreseeable, the company failed to use reasonable safeguards, and a third party exploited that failure. The measure targets harms associated with malicious use or “jailbreaking,” rather than making companies automatically liable whenever a user misuses a tool.
City contracts face a 24-hour reporting rule
Introduction 2601 would require city contractors and the agencies that hire them to notify the Office of Cyber Command in writing within 24 hours after learning of an AI safety incident related to a covered contract. The office would then have to publicly disclose the occurrence of a reported incident within 24 hours. A separate proposal would require the city to plan for AI-related disruptions to its information systems, infrastructure, government operations or public safety.
The package also includes measures on AI company disclosures, chatbot privacy and transparency, deepfakes, and the effect of algorithmic tools on city workers. One proposal would require agencies to report job displacement, salary changes tied to altered responsibilities and training prompted by algorithmic tools. Together, the bills reach beyond model developers to city agencies, contractors, validators and businesses deploying AI.
October 5 hearing will test the proposal’s reach
The Council says its Committee of the Whole hearing on October 5 will convene all 51 members to examine AI risks. Menin has asked the leaders of Anthropic, OpenAI, Google, SpaceXAI and Meta to participate and says the Council may use its subpoena powers if necessary. The hearing is the next public test of the package; the measures’ scope, enforcement and whistleblower-payment details remain matters for lawmakers to consider.
“We can and must be both pro-innovation and pro-safety.”
Julie Menin, New York City Council Speaker