More than 1 in 10 chance AI ‘could kill all humans,’ says Anthropic safety lead after colleague quits
More than 1 in 10 chance AI ‘could kill all humans,’ says Anthropic safety lead after colleague quits. A senior Anthropic safety researcher has said there is more than a 10 percent chance artificial intelligence "could kill all humans" by the end of the decade, just hours after a colleague resigned over fears the AI lab and its rivals are carelessly racing to build "superhuman systems" they cannot control. In a post on […]
What happened
Image source, Getty ImagesImage caption, Anthropic makes the AI tool Claude, which is used as a chatbot and as a tool to help codingByTom GerkenTechnology reporterPublished9 September 2026A top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a greater than 10% chance it "could kill all humans" within the next decade. After apocalyptic warnings about the threats posed by AI, leaders like Sam Altman and Elon Musk backed Anthropic CEO Dario Amodei’s calls to ‘slow the pace’Facing a public uproar over Anthropic researchers’ repeated warnings that artificial intelligence could kill all of humanity by 2030, the AI company’s CEO Dario Amodei issued a proposal at the weekend to slow down the technology’s advancement to ensure public safety.
This was followed by an admission from a senior Anthropic researcher, who said he believes there is a greater than 10% chance AI will "kill all humans" within the next decade. On Tuesday night, a senior employee at Anthropic admitted he believed there was a greater than 10% chance the technology could “kill all humans” in the next decade and warned that his AI company did not have a plan to ensure ASI was aligned, meaning it did no harm.
Photograph: Dado Ruvić/ReutersAnalysis‘Too little, too late’: critics perplexed and suspicious of AI leaders’ call for a slowdownRobert Booth UK technology editorFrom the Trump administration to AI experts, plans by the Anthropic boss to boost safety have spawned a largely negative response OpenAI boss and Elon Musk back slowdown on ‘reckless’ AI development Editorial: controlling AI – humanity cannot outsource its survival The safety debate that ignited when whistleblowers last week warned that the most advanced AI poses an existential threat to humanity should come with a health warning: side effects include whiplash.
Leading figures in the AI field have been raising the alarm about the safety threat the tech poses for years, with the heads of OpenAI, Google Deepmind and Anthropic saying as much in 2023. At the weekend the New York Times reported that Barack Obama on Thursday privately urged the Democrats to prioritise AI oversight in their agenda, telling party leaders on Thursday: “This is something that is moving very fast in private hands, and if we don’t get on top of it, I think can be dangerous. ”The former British prime minister Rishi Sunak, who advises Anthropic, wrote on Sunday: “We have had a wake-up call to take AI safety more seriously, to work out how to set limits on research.
The wider picture
Skip to main contentSkip to navigation A safety expert at Anthropic believes there is a greater than 10% chance that AI could spark an event that wipes out humanity. Might there even be a greater than 10% chance that AI could “kill all humans” in the next decade? By Wednesday, cutting-edge AI started to look a lot less appealing when the resigning Anthropic researcher Jacob Coxon warned “the people building AI earnestly believe that it could kill us all by the end of the decade”.
Another criticism of Amodei’s call for “pacing” of AI progress to allow safety measures and guardrails to keep up has came from Prof Stuart Russell, one of the world’s leading AI experts. Separately, the Financial Times reported, external Anthropic withheld its latest model from the UK's AI Security Institute (AISI), one of the leading bodies in the world for assessing AI risk. In Anthropic's safety report from August, external, it wrote there was a low risk of its models becoming misaligned with a hypothetical powerful organisation's desires, causing it to exploit or tamper with its systems.
House Speaker Mike Johnson likewise said Congress would not lead the charge on regulating AI safety, laying the responsibility at the feet of AI companies. But the Cabinet Office – the part of government which leads on AI safety – said the UK "cannot simply turn AI off".
What has been reported
A kill switch act is also being debated by US politicians but President Trump has said he does not believe the technology is a threat to humans – instead calling the industry a "golden goose". Photograph: Dragon Claws/AlamyOpenAI boss and Elon Musk back calls to put brakes on ‘reckless’ AI developmentRare show of unity from rival developers after safety warnings from Anthropic boss and AI researchers Analysis: critics perplexed and suspicious of call for a slowdown The Guardian view: humanity cannot outsource its survival Sam Altman and Elon Musk have backed a call from the head of Anthropic, Dario Amodei, to “slow the pace” of AI development after he warned that an AI swarm could otherwise become capable of “taking over the entire internet” within a year.
In a rare show of unity, the rival technology leaders threw their weight behind the appeal from the founder and chief executive of the artificial intelligence company Anthropic, which came after a series of warnings from AI researchers last week. One session is set to be called “prudence at the frontier”, which is expected to explore the possibility for clear red lines on the uses of AI to create bioweapons and launch cyber-attacks on critical systems. skip past newsletter promotionafter newsletter promotionThose expected to attend the event overseen by the Ditchley Foundation, which has drawn up a draft charter for AI leaders, include: senior executives at OpenAI and Anthropic; the Nvidia chief executive, Jensen Huang; Hassabis; Niccolo de Masi, the chair and chief executive of IonQ,; and Paolo Benanti, an adviser to the pope on issues of AI and technology ethicsThe OpenAI researcher Aidan McLaughlin called Amodei’s essay “excellent” and said he agreed “with basically every word”.
Amodei said his company would “unilaterally” commit to the first of the steps, saying Anthropic would provide “third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training”. 4% in July boosted by AIAnthropic details bad actors’ efforts to misuse its AI for bioweaponsMore Anthropic researchers warn of AI’s perils but Musk dismisses ‘psyop’OpenAI not on track to reduce risk of ‘catastrophic’ loss of control, says board memberWe have started losing control of AI.
Illustration: Ales Utouka/AlamyAI could kill all humans in next decade, warn experts: but how seriously should we take them? Beatrice Fihn, who won the 2017 Nobel peace prize for leading the International Campaign to Abolish Nuclear Weapons, told parliamentarians: “It is not the first time we’re confronted with abilities that could end up killing us all. ”The session on Monday was convened by Control AI, a lobbying group pushing for international regulation of the technology.
What happens next
Evan Hubinger, alignment science lead at Anthropic, posted the comment after Jacob Coxon, a 28-year-old researcher at the San Francisco company and previously at OpenAI, resigned, claiming “neither company was acting responsibly”. A spokesperson added: “The AI Security Institute continues to collaborate closely with industry partners, including Anthropic. ”View image in fullscreenThe Labour MP Darren Jones has written to world leaders to call for a multinational treaty to ensure AI is developed safely. He has dedicated part of his fortune to campaigning for AI safety and says some senior AI executives would be happy for humanity to be wiped out.
Despite being an early investor in Anthropic and Google DeepMind, two of the leading AI labs, Tallinn estimates that 10-15% of AI employees believe the technology will be a worthy successor to humanity. Skip to main contentSkip to navigation The easiest way not to build super-intelligence is for Anthropic to agree not to build it, said David Sacks, co-chair of Trump’s council of advisers on science and technology. Yes, he was talking about artificial superintelligence – an as-yet-nonexistent version of the technology that outcompetes humans in most domains – but his comments, and the chorus of experts that echoed him, jangled plenty of nerves.
Another lurch came when Anthropic revealed it had found users dodging controls to use its existing models “in ways that could support biological weapons development” – think novel mosquito-borne or bird flu viruses.
The report has been compiled by The Daily Waves using information reported across theguardian.com, bbc.co.uk. Details are presented according to the information available at the time of publication and may change as authorities, organisers or other relevant parties provide updates.