Anthropic researchers say AI could cause human extinction by 2030
Anthropic researchers say AI could cause human extinction by 2030. One researcher who resigned said many in industry believe that without regulation AI could kill off humanityArtificial intelligence could kill off humanity within the decade, according to three researchers with the industry giant Anthropic, one of whom has quit his job in protest. The latest doom-laden predictions came in posts on social media on Tuesday by a researcher who said he resigned because Anthropic and his previous employer, OpenAI, were ignoring or at best mishandling their response to the threat. Continue reading. . .
What happened
After apocalyptic warnings about the threats posed by AI, leaders like Sam Altman and Elon Musk backed Anthropic CEO Dario Amodei’s calls to ‘slow the pace’Facing a public uproar over Anthropic researchers’ repeated warnings that artificial intelligence could kill all of humanity by 2030, the AI company’s CEO, Dario Amodei, issued a proposal at the weekend to slow down the technology’s advancement to ensure public safety. Figure caption, 'There is a possibility of human extinction,' former Anthropic employee says about AIByBrandon Livesay and Mitchell Labiak, business reporterPublished13 September 2026An artificial intelligence researcher who quit AI firm Anthropic has said people working on the technology were "genuinely frightened" about the speed of its advancements and what it could mean for humanity.
Anthropic CEO Dario Amodei called for a coordinated slow down of AI development over the weekend, after researchers warned recently that AI model progress could outpace our ability to safely deploy increasingly complex systems and verify and control the actions of AI agents. While Microsoft isn’t one of the top AI providers yet, Suleyman also told The Verge earlier this year that it’s the company’s goal to “prove that we can become one of the top four labs in the world. ” Microsoft is currently working on models that will compete with Google, Anthropic, and OpenAI, and it’s committing, as part of its code of conduct, that its models should not exceed human control.
These incidents, and others, have led some researchers to call for a slow down on AI model progress, while simultaneously racing to solve Millennium Prize problems and create superintelligence. ”[Humanist AI] rejects the race to produce an all-purpose superintelligence that could evade these safeguards,” says Microsoft. “We are building something fundamentally useful and safe even if that means compromising on ultimate generality, autonomy or capability. ”Microsoft is also committing that its own models won’t communicate in “any form beyond simple human understanding, either in their chain of thoughts or with other agents or AI systems. ” Models can be made to show their reasoning as they work, allowing researchers or automated systems to monitor what they’re doing.
That is how we operate everyday. ”Beyond the human control concerns, Microsoft’s AI code of conduct also commits to its own models discouraging “patterns of interaction that cause excessive reliance or emotional dependence,” a clear reference to the problem of sycophancy in AI models, where chatbots prioritize pleasing users over providing honest or accurate responses.
The wider picture
Photograph: Dado Ruvić/ReutersAnalysis‘Too little, too late’: critics perplexed and suspicious of AI leaders’ call for a slowdownRobert Booth UK technology editorFrom the Trump administration to AI experts, plans by the Anthropic boss to boost safety have spawned a largely negative response OpenAI boss and Elon Musk back slowdown on ‘reckless’ AI development Editorial: controlling AI – humanity cannot outsource its survival The safety debate that ignited when whistleblowers last week warned that the most advanced AI poses an existential threat to humanity should come with a health warning: side effects include whiplash.
Amodei’s plan requires US government action and Trump said last week he had no concerns about AI leading to human extinction. Anthropic researcher believes more than 10% chance AI 'could kill all humans'Published5 days agoMany others in the industry have also voiced their concerns since Coxon's social media post, including Anthropic scientist Evan Hubinger. Former Prime Minister Rishi Sunak, who is a paid adviser to Anthropic, wrote in the Sunday Times, external that he too was worried about the risk of AI to humanity despite generally being optimistic.
The findings, made in the firm's recent threat intelligence report, is the latest in a growing series of warnings from AI researchers and industry insiders about the technology's potential risks to humanity. Why some experts increasingly fear AI will take overPublished4 days agoAnthropic researcher believes more than 10% chance AI 'could kill all humans'Published5 days agoThe report indicated that cybercriminals and state-backed hackers have increasingly used its technology to assist their operations. Figure caption, Watch: "Slow it down" – Bernie Sanders proposes bill to ban AI superintelligenceThe report – the company's first this year – comes after a top safety researcher at Anthropic warned AI is advancing so quickly he believes there is a greater than 10% chance it "could kill all humans" within the next decade.
Skip to main contentSkip to navigation CEO of Anthropic Dario Amodei, speaks in India in February.
What has been reported
Photograph: Bhawika Chhabra/ReutersView image in fullscreenCEO of Anthropic Dario Amodei, speaks in India in February. While lawmakers and the tech industry consider how to address growing fear and distrust of AI, many of the hurdles impeding comprehensive AI regulation remain the same as before last week’s apocalyptic declarations from a young Anthropic researcher. View image in fullscreenElon Musk, CEO of Anthropic Dario Amodei, and OpenAI CEO Sam Altman have said AI firms should slow down the development of the powerful technology.
Amodei’s pacing proposal has drawn heavy skepticism from critics and researchers who allege that AI firms are attempting to preempt more stringent government regulation and retain the status quo of power within the industry. Anthropic and OpenAI did not immediately respond to a request for comment. The AI industry, especially the factions within it that focus on whether the technology is aligned with humanity’s interests, has also long warned about its existential threats.
OpenAI, for instance, shut down a “superalignment” team in 2024 dedicated to studying long term AI risks and earlier this year disbanded another team intended to communicate how OpenAI would benefit humanity. Explore more on these topicsAI (artificial intelligence)ComputingAnthropicOpenAISam AltmanElon MuskfeaturesShareReuse this contentMost viewedMost viewed
What happens next
NewsAITechMicrosoft says ‘people matter more than AI’ following safety concernsMicrosoft has created a ‘humanist AI code of conduct’ in response to safety concerns. Microsoft has created a ‘humanist AI code of conduct’ in response to safety concerns. by Tom WarrenSep 14, 2026, 1:00 PM UTCImage: Cath Virginia / The VergeTom Warren is a senior correspondent and author of Notepad, who has been covering all things Microsoft, PC, and tech for over 20 years. Microsoft is publishing a 37-page “humanist AI code of conduct” today, amid growing safety concerns over AI model progress.
Microsoft’s AI code of conduct makes it clear that “people matter more than AI,” and that AI models are not conscious and “should not be designed to imitate consciousness. ” Microsoft also rejects “the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights. ”That’s a direct swipe at AI welfare research and model consciousness — concepts Anthropic has been pushing hard on lately. Amodei said earlier this year that Anthropic is “open to the idea” that models could be conscious, and Anthropic seems to believe chatbots might already be thinking, feeling entities.
In response, Microsoft AI CEO Mustafa Suleyman called Anthropic’s speculation “really, really dangerous” during an episode of Decoder in June. “Models should remain subordinate to humanity, subject to meaningful human oversight and control,” says Microsoft.
The report has been compiled by The Daily Waves using information reported across theguardian.com, theverge.com, bbc.co.uk. Details are presented according to the information available at the time of publication and may change as authorities, organisers or other relevant parties provide updates.