Wednesday briefing: Why tech companies might be only too happy for us to believe AI will ‘kill us all’
Wednesday briefing: Why tech companies might be only too happy for us to believe AI will ‘kill us all’. Photograph: Manuel Orbegozo/ReutersWednesday briefing: Why tech companies might be only too happy for us to believe AI will ‘kill us all’In today’s newsletter: It is hard to tell fact from fiction when it comes to AI.
What happened
In today’s newsletter: It is hard to tell fact from fiction when it comes to AI. What is really going on – and what should the government do about it? Good morning. As a general rule, it pays to be suspicious of any gigantic company that claims it’s developing a tool capable of destroying humanity. But in recent days, a number of warnings from the AI industry have suggested that even tech insiders are starting to worry about what they have unleashed. In a lofty essay published on Saturday, Dario Amodei, the founder of Anthropic (the company behind Claude), argued that tech companies need to “slow” the pace at which they’re developing the newest and most sophisticated AI models. Lucy Letby | Three babies might have survived if hospital had acted upon concerns over Lucy Letby, an inquiry has found. Lady Justice Thirlwall condemned the ‘complete failure’ to protect babies on neonatal unit at Countess of Chester hospital. UK politics | The leader of Reform UK in Wales stood down after arrest on suspicion of assault. Dan Thomas, a former Tory councillor, was elected to Senedd as leader of the opposition in May. AI | The progressive senator Bernie Sanders and rightwing strategist Steve Bannon have called for restrictions on artificial intelligence (AI) but offered competing visions for what they termed a “cold war” with China. UK news | Young people in Rotherham face a local jobs market with the fewest suitable opportunities in Britain, according to a report that warns stark regional divisions are fuelling a crisis in youth work. Defence | John Healey is in talks with the Canadian government about joining a new global defence bank intended to help allies rearm to counter mounting security threats, just weeks after Rachel Reeves rejected the move. Continue reading. . .
By Wednesday, cutting-edge AI started to look a lot less appealing when the resigning Anthropic researcher Jacob Coxon warned “the people building AI earnestly believe that it could kill us all by the end of the decade”. In a lofty essay published on Saturday, Dario Amodei, the founder of Anthropic (the company behind Claude), argued that tech companies need to “slow” the pace at which they’re developing the newest and most sophisticated AI models.
Tech companies often tell fanciful stories about the fearsome potential of their products to convince the public (and the stock market) of those capabilities. It was published just two days after Jacob Coxon, a 27-year-old researcher at Anthropic, quit his job and posted an alarming thread on X warning that “the people building AI earnestly believe that it could kill us all by the end of this decade”. Apparently, AI firms now believe the risks posed by their technologies are too great for business to continue as usual.
It was only seven days ago that AI enthusiasts were cheerily toying with OpenAI’s newest model GPT-6 Astra which was marketed in part as a lifestyle tool to help people book tennis courts, run fashion companies and order takeaways.
The wider picture
Until now, such watchdogs have only been brought in at the companies’ behest or given limited access to systems to investigate when things go wrong. After apocalyptic warnings about the threats posed by AI, leaders like Sam Altman and Elon Musk backed Anthropic CEO Dario Amodei’s calls to ‘slow the pace’Facing a public uproar over Anthropic researchers’ repeated warnings that artificial intelligence could kill all of humanity by 2030, the AI company’s CEO, Dario Amodei, issued a proposal at the weekend to slow down the technology’s advancement to ensure public safety.
While AI companies have made grandiose claims about the threats and benefits from their technology, they have also historically provided little transparency or oversight into how their models operate and spent millions in lobbying efforts to shape favorable legislation. But in recent days, a number of warnings from the AI industry have suggested that even tech insiders are starting to worry about what they have unleashed. Elon Musk and Sam Altman, the head of OpenAI, voiced support for Amodei’s arguments, while Donald Trump, who has staked America’s economy on AI, described the worries as a “HOAX”, and insisted the only control on AI the world needs is a “STRONG AND SMART (High IQ! ) PRESIDENT”.
For this morning’s newsletter I spoke to Aisha Down, the Guardian’s global technology reporter, to try to parse the reality of AI from the unreality of the AI discourse, and to figure out what the government should do about it. Five big stories Lucy Letby | Three babies might have survived if hospital had acted upon concerns over Lucy Letby, an inquiry has found. Photograph: Bloomberg/Getty ImagesThe most significant detail in Amodei’s essay, We Must Pace The Frontier, was a reference to the Hugging Face incident, which occurred in July, when the rival tech company OpenAI revealed that an autonomous AI agent powered by its technology hacked a prominent database of AI models, Hugging Face.
What has been reported
But tech researchers have since said that OpenAI turned off the model’s safety mechanisms, gave it impossible tasks to complete and ran it 1,200 times. “That’s human decision-making,” the tech research Eryk Salvaggio wrote in an essay on Substack. But industry critics are sceptical about what this incident can tell us, particularly since the only people who have examined it did so at OpenAI’s discretion. This reflects a broader problem where companies that claim to be developing products capable of catastrophic damage get to mark their own homework.
These self-styled whistleblowers don’t tend to leak documents that reveal useful information about the inner workings of AI companies, so their warnings of imminent apocalypse instead fuel speculation about the perils (and power) of AI. The AI narrative is dominated by what the tech scholar Lee Vinsel calls “criti-hype” – criticism that feeds, and is fed by, hype. While China tends to compare AI to “electricity,” western AI companies compare it to the atom bomb, which makes Anthropic and its rivals seem less like companies making everyday decisions about how they choose to build particular tools, and more like Promethean stewards of a fourth Industrial Revolution.
But there are a number of ways that Anthropic (and its competitors) might gain from a slowdown.
What happens next
An industry-orchestrated slowdown could freeze the pace of AI development in China and might also help firms like Anthropic avoid the possibility of stringent government regulations by putting them on the front foot. One of the people who has called this out is David Sacks, co-chair of Trump’s council of advisers on science and technology, who said in response to Amodei’s essay: “demanding your preferred regulatory framework … will look like blackmail of the public and the political system”.
And many of the organisations that are supposed to make AI safer are closely aligned with the tech industry, such as Britain’s AI Security Institute, which receives funding from the industry, and doesn’t have the legal power to regulate AI or to compel developers to hand over their models. But as Aisha makes clear, the design of technology is a choice, not an inevitability, so there’s an argument for stubborn resistance to the idea there’s no alternative.
The Times says “Failure to act allowed Letby to kill three babies”, the Mirror writes “They failed and babies died”, and the Express has “Cot cams after ‘complete failure to protect babies’”. The Telegraph says “Mayors to control water companies”, the i Paper leads on “Russian spies ‘recruiting UK teenagers’”, and the FT writes “Fresh test for Healey as state pension poised to cross income tax threshold”. The Latest Photograph: The GuardianLucy Letby inquiry finds ‘some babies would have been saved’ if hospital acted soonerThree babies might have survived and others could have been protected if hospital staff had taken action over concerns about the nurse Lucy Letby, an inquiry has found.
The report has been compiled by The Daily Waves using information reported across theguardian.com. Details are presented according to the information available at the time of publication and may change as authorities, organisers or other relevant parties provide updates.