Emerging
Published Sep 14, 2026Updated 2h ago Major3
100%
Anthropic Cofounder Calls for Mandatory AI Kill Switch
Jack Clark, a cofounder of AI firm Anthropic, has called for mandatory independent verification of AI "kill switches" that would allow governments to shut down dangerous systems. The proposal emerges amid industry warnings about AI risks and US legislative efforts, though President Trump has rejected calls to slow AI development.
Quick Facts
- Clark called for mandatory AI kill switches with independent third-party verification
- Amodei renewed calls to slow AI development and increase monitoring
- US lawmakers proposed the Kill Switch Act requiring AI shutdown mechanisms
- AI researcher Jacob Coxon quit Anthropic over extinction concerns; post went viral
- Trump rejected calls to slow AI development via social media
Related Insights
Anthropic warning: humans may need “kill switches” to shut down rogue AI systems | BBC News BBC News · YouTube
Trump calls AI safety fears a 'hoax' and rejects new safeguards Cruxwire · YouTube
Mandatory 'Kill Switch' for AI Systems May Be Needed, Anthropic Founder Suggests TheReelyNews · YouTube
US confirms weapons in orbit, Anthropic's kill-switch call, pension up £488, steel nationalised Will · YouTube
🚨 BREAKING: Trump EXPOSED AI Safety as Hoax (2025) WirePulse Daily · YouTube
Jack Clark, one of seven founders of Anthropic, has called for mandatory independent verification of AI "kill switches"—mechanisms to shut down artificial intelligence systems if they become dangerous. In interviews with the BBC, Clark said most major AI labs already possess such shutdown capabilities, but lawmakers may need to enforce their existence through regulation.
Clark framed the issue as part of a broader policy conversation around AI safety. "Should you mandate for companies to definitely have a kill switch? Is that kill switch verifiable by a third party?" he asked. "I think that's the kind of thing society is going to want to know and might want to eventually pass rules around." He emphasized that allowing AI to develop as a "totally unregulated industry" was inadvisable, stating: "We are rolling dice with immense risks. And the point is, we have to change the course of this industry."
The call comes amid escalating public debate over AI risks. Anthropic CEO Dario Amodei recently reiterated calls to slow AI development and increase monitoring, though he stipulated that safeguards should be introduced "without sacrificing commercial advantage." Last week, a post by an AI researcher who left Anthropic over extinction concerns went viral. In response, Anthropic scientist Evan Hubinger stated he personally assessed the probability of human extinction from AI within the next decade at over 10%. Computer scientist Geoffrey Hinton, a Nobel laureate known as the "Godfather of AI," told the BBC that a 10% extinction probability "is not unreasonable."
US lawmakers have proposed the Kill Switch Act, which would require companies to possess shutdown mechanisms and grant government agencies power to order AI tools disabled or restricted. However, US President Donald Trump has rejected calls to slow AI development, stating on social media that claims of AI destroying humanity are a "hoax" and warning against any measures that might cede advantage to China.
Some industry figures dispute the urgency of extinction risks. Clement Delangue, leader of developer platform Hugging Face, and George Arison, leader of Grindr, have suggested that existential risk warnings may be overstated or leveraged for commercial purposes. Arison argued that extreme valuations—Anthropic is valued at $965 billion—require companies to claim they will dominate entire industries. OpenAI CEO Sam Altman announced the company would not pursue a public offering this year, citing ongoing AI safety debates, though Anthropic continues preparing for an IPO.
The UK government has rejected establishing an AI kill switch, with a spokesperson stating it "would not prevent the technology being developed or misused elsewhere."
Why This Matters
The proposal would require AI companies to demonstrate independently verified shutdown mechanisms, potentially affecting major AI labs' compliance costs and product rollouts. The Kill Switch Act would grant US agencies power to order AI tools disabled or restricted. Anthropic is valued at $965 billion and preparing for an IPO, while OpenAI has announced no public offering this year. The UK government has rejected establishing a domestic kill switch. AI researcher departures and extinction-probability estimates (over 10% within a decade by an Anthropic scientist, called 'not unreasonable' by Geoffrey Hinton) have intensified public debate, with figures such as Clement Delangue and George Arison disputing the urgency. US President Donald Trump has rejected slowing AI development, warning against ceding advantage to China.
Timeline & Sources
Jan 1, 2021
WireAnthropic founded by former OpenAI employees