FINAL HOUR: Get InvestingPro for less than $2 per week before our sale ends TODAY

FINAL HOUR: Get InvestingPro for less than $2 per week before our sale ends TODAY. To the public, the idea that the technology could perform many of the tasks it can today, much less become super intelligent and take over the world, felt largely hypothetical, if not entirely a thing of science fiction.

What happened

Photograph: Michael Dwyer/APAnalysisWhy a decade of doomsday warnings failed to slow the AI raceJohana BhuiyanFrom Stephen Hawking to Jacob Coxon’s viral Anthropic resignation, fears that AI could threaten humanity have shaken the industry without stopping its pursuit Before an Anthropic researcher resigned and declared human extinction imminent last week, tech leaders and scientists had sounded the alarm about a superintelligent AI ending humanity for over a decade. The development of artificial intelligence “could spell the end of the human race”, warned professor and astrophysicist Stephen Hawking in 2014 – a little less than a decade before the public got its hands on the generative AI features of the original version of ChatGPT.

It’s a now-familiar warning, echoed across the AI and tech industry by countless employees and CEOs of the leading frontier labs across the US. Days later, the CEOs of the leading US AI firms answered a call from Anthropic CEO Dario Amodei to slow down what he called a “reckless” approach to creating AI. A race to the bottom, spurred by commercial incentives, can make these risks more acute. ”While there’s often scant detail and even less hard evidence provided for the ways AI could lead to the ultimate doomsday scenario, that specter has loomed large over the technology’s development.

Gebru, who founded the independent research firm the Distributed AI Research Institute, still argues that the so-called doomer scenarios are a distraction from the real-time harms that AI is causing today. Two weeks after the 2026 Hugging Face breach, OpenAI announced it was pausing development of some aspects of AI training.

The wider picture

Skip to main contentSkip to navigation ‘Been thinking a lot about whether it’s possible to stop humanity from developing AI,’ OpenAI CEO Sam Altman wrote to Elon Musk in 2015. However, at the time, generative AI was still in its infancy. Teams of researchers and tech leaders issue somber warnings about the dangers of AI built irresponsibly. Rather than push for a pause on any development, a faction of researchers breaks away to create a new AI company.

However, no moment has created as many reverberations across the industry as the now-viral resignation of Anthropic safety researcher, Jacob Coxon, who tweeted on 9 September that Anthropic and OpenAI were “racing straight to self-improving super intelligence and gambling with our lives”. His concerns were echoed by dozens of staffers from Anthropic, OpenAI and other competing firms. “AI brings risks, and because it is such a powerful technology, these risks are serious,” Amodei wrote in a blog post.

They include the risk of losing control of AI systems, misuse of AI for cyber-attacks and bioterrorism, and serious economic disruption.

What has been reported

Hawking’s fear was that AI would become so advanced that it would redesign itself, and humans “who are limited by slow biological evolution” wouldn’t be able to compete “and would be superseded”, he said in an interview with the BBC. Google was the biggest household name in the field at the time, particularly after acquiring a two-year-old AI startup, DeepMind, for $650m in January 2014. But Google and other firms like it weren’t building AI responsibly enough, according to Elon Musk and Sam Altman.

The threat that AI could be developed in a way that could be dangerous for humanity was so pervasive, they claimed, that they co-founded a nonprofit dedicated to building it safely, OpenAI, in 2015. OpenAI’s stated mission was to “advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return”. But even before there were clear consumer use cases for more rudimentary versions of the technology, both Musk and Altman were already motivated by competition.

“Been thinking a lot about whether it’s possible to stop humanity from developing AI. I think the answer is almost definitely not,” Altman wrote in 2015 in an email to Musk, which was released as part of Musk’s 2024 lawsuit against OpenAI.

What happens next

“If it’s going to happen anyway, it seems like it would be good for someone other than Google to do it first. ”In the intervening years, priorities within OpenAI shifted away from safety, some employees claimed. In 2021, several employees focused on AI ethics and alignment at OpenAI left the firm. They argued the company had started to prioritize rapid commercial development over safety. Those employees created another company: Anthropic. Founded by siblings Dario and Daniela Amodei, Anthropic pitches itself as an “AI safety and research company” with a familiar mission: Developing and deploying AI models “in a way that benefits people”.

The underlying premise of each of these firms is that AI is “only safe in their hands” and that if they don’t build it, someone else will, said Sarah Myers West, co-executive director of the AI Now Institute, which studies the impacts of AI. That paranoia is fueling a race to the bottom, she argued. skip past newsletter promotionafter newsletter promotionBoth Anthropic and OpenAI face enormous pressure from investors to turn a profit soon.