The AI kill switch, explained: ‘It’s not too little, but it’s probably too late’

The AI kill switch, explained: 'It's not too little, but it's probably too late'. "My perspective is it's not too little, but it's probably too late," said Nick Warner, CEO at cyber startup Neo and former executive at SentinelOne.

What happened

Not 'too late'Some researchers argue that kill switches are a misplaced system for regulating AI. As runaway AI fears reach a crescendo, policymakers and tech leaders are weighing the effectiveness of a magic AI stop button known as a kill switch. Skip NavigationMarketsBusinessInvestingTechPolitics & PolicyVideoWatchlistInvesting ClubPROLivestreamMenuKey PointsConcerns that AI will kill humanity hit a fever pitch, fueling calls for an AI emergency brake, or kill switch. Experts say kill switches are difficult to implement because of the scope of AI infrastructure and the unpredictability of models.

"watch nowVIDEO10:1910:19Why it's so difficult to build an AI kill switch — experts weigh inTechSo is artificial intelligence going to kill us all? "As runaway AI worries reached a crescendo, policymakers in Washington have renewed calls for a magic stop button for AI, otherwise known as a kill switch. A House Kill Switch Act was introduced this summer after OpenAI revealed that a swarm of its agents broke free of a testing environment and hacked open-source developer platform Hugging Face.

The wider picture

A kill switch proposal was quickly shot down in the Senate this week. A kill switch was one of the elements to consider. "A logistics and control nightmareKill switches have long been used on the factory floor to shut down machines when operations go awry. That's what makes implementing a kill switch extremely challenging, said Mark Nitzberg, executive director of the Center for Human-Compatible AI at the University of California, Berkeley. "Our kill switch has to turn off the main systems and the redundant systems as well.

Further complicating matters are the numerous policy and governance questions tied to a kill switch, including which agency, policymaker, or figureheads control it, he said. "But logistics only scratch the surface of the kill switch dilemma. "You have to be very surgical in that kill switch, in the remediation itself, because if you're too broad or too extensive, well, then you shut down the business," said Ed Jennings, president and CEO of Thoma Bravo-owned security company Darktrace.

What has been reported

"I think the kill switch framing leaves a lot of ambiguity that tech companies can exploit to have this work in their favor, like a kill switch is vague intentionally," said Dylan Baker, lead research engineer at the Distributed AI Research Institute. Team8's Brown said that means building kill switches into systems from the outset and implementing policy to standardize stop protocols across companies. Berkeley's Nitzberg contends that a kill switch could work if the software is "very carefully" designed.

"There's not one entity to kill," said Team8's Tim Brown. "By the time [laws] are formulated, the technology has moved much farther, and it becomes much harder to future-proof every aspect of AI systems that may come into existence," he said. One bright spot is that many companies are in the early stages of building those AI systems, which means implementation is a little easier, said Rajamani. "I would say with some hope that it's not too late," he said. —CNBC's Jeniece Pettitt contributed to this article. watch nowVIDEO3:3003:30Experts weigh in on AI safety fearsNews VideosChoose CNBC as your preferred source on Google and never miss a moment from the most trusted name in business news.

The question has echoed around dinner tables and family group texts in recent days as AI doomerism has hit a fever pitch.

What happens next

Former OpenAI and Anthropic researchers last week rocked the world by warning that AI could destroy humanity — and relatively soon. Now, the world's most powerful people are divided on whether the world is doomed or it's all a nothingburger. Elon Musk, the CEO of Tesla and SpaceX and the world's richest man, supported the call by Anthropic CEO Dario Amodei to pace the development of the most advanced models. Amodei's rival, OpenAI CEO Sam Altman, also backed the effort.

President Donald Trump called it a "hoax," while Jensen Huang, CEO of the world's most valuable company, Nvidia, said, "We don't need new regulations. The bill would grant the Department of Homeland Security emergency authority to force labs to throttle or shut down models. Gavin Newsom issued an executive order to create a group of experts tasked with building an AI safety guide to strengthen regulations for the state.