‘Godfather of AI’ says tech regulation is nearing Covid-style pivot moment

‘Godfather of AI’ says tech regulation is nearing Covid-style pivot moment. Photograph: Atlantic Studios‘Godfather of AI’ says tech regulation is nearing Covid-style pivot momentSafety crisis makes it more likely that governments will be spurred into action, says Yoshua BengioConcerns over AI safety are reaching a point where governments realise they must act to protect the public, similarly to in the Covid pandemic, according to one of the “godfathers” of the technology.

What happened

Safety crisis makes it more likely that governments will be spurred into action, says Yoshua BengioConcerns over AI safety are reaching a point where governments realise they must act to protect the public, similarly to in the Covid pandemic, according to one of the “godfathers” of the technology. Yoshua Bengio said recent events, including a “swarm” of OpenAI agents hacking a startup and tech insider warnings of an existential threat, were cutting through – making government action more likely. Continue reading. . .

Explore more on these topicsAI (artificial intelligence)ComputingnewsShareReuse this contentMore on this storyMore on this story‘If you’re building Frankenstein, stop’: JD Vance dismisses calls for AI regulationTrump facing AI backlash in Congress as push for guardrails intensifiesFull StoryAI isn’t going to end humanity. . . right? – Full Story podcastCould AI really wipe out humanity – six experts spell out the risksEurope must build own AI or risk getting cut off by US or China, says ECB’s LagardeAI-linked stocks slide after tech bosses call for slowdown in ‘reckless’ developmentOpenAI urges UK lawmakers to rein in technology amid growing safety fears OpenAI boss and Elon Musk back calls to put brakes on ‘reckless’ AI developmentUK lawmakers urge Burnham to back ban on superintelligent AI after chilling warningsAnthropic details bad actors’ efforts to misuse its AI for bioweaponsMost viewedMost viewed

Yoshua Bengio said recent events, including a “swarm” of OpenAI agents hacking a startup and tech insider warnings of an existential threat, were cutting through – making government action more likely. Comparing the AI safety crisis to the onset of the Covid pandemic in 2020, Bengio said: “Think about how quickly governments moved after the beginning of the pandemic when they realised that public safety, their future, democracy, was in danger. So we are, I think, nearing that point. ”Concern over the potential threat of powerful AI systems has reached a new pitch in recent months after a series of safety incidents involving OpenAI and Anthropic agents carrying out unsanctioned activities such as hacking third parties, hijacking a German website, and using fake identities to try to trick developers.

Bengio, a professor at the University of Montreal, earned the “godfather of AI” moniker after winning the 2018 Turing award, seen as the equivalent of a Nobel prize for computing. The LawZero technology is viewed by Bengio as a counterpoint to reinforcement learning, a trial-and-error development technique used by major AI companies where AIs are rewarded for working out how to carry out a specific task.

The wider picture

The Canadian computer scientist believes, along with other experts, this training technique encourages AIs to pursue their goals recklessly and cause harm in the process, as shown by the OpenAI “swarm”. Deployed alongside an AI agent, Bengio’s technology would raise potentially harmful behaviour by an autonomous system – having weighed whether its actions could cause harm. Bengio expects the technology to be deployed for other uses eventually, such as accelerating scientific breakthroughs. Skip to main contentSkip to navigation Bengio is a prominent voice in the campaign to rein in breakneck AI development.

Photograph: Atlantic StudiosView image in fullscreenBengio is a prominent voice in the campaign to rein in breakneck AI development. The Canadian computer scientist, a prominent voice in the campaign to rein in breakneck AI development, said he was now “more optimistic than many observers because I see the public moving”. His comments came as 42 fellows and foreign members of the Royal Society wrote to the organisation’s president, Sir Paul Nurse, to express their “extreme concern” over the pace of AI development.

“By the time the situation becomes obvious to the wider public, it may be too late to act,” the researchers wrote in an open letter to Nurse.

What has been reported

“We believe this is an emergency, and call on the Royal Society to use its influence to convey this view to government and the media. ”Last week, a researcher at Anthropic, the startup behind the Claude chatbot, resigned after warning that colleagues believed AI could “kill us all by the end of the decade”. Days later, Anthropic’s chief executive, Dario Amodei, called for a slowdown in the pace of cutting-edge AI development – a move that was immediately supported by OpenAI, Google and Elon Musk, the CEO of SpaceX.

However, sceptics of the “slowdown” call have claimed that calls by major AI firms are an example of “regulatory capture”, where companies persuade governments to introduce safety regimes that raise the cost of smaller rivals being able to compete. They also claim existential fears overshadow more immediate issues with AI such as its impact on copyright-protected work and human rights. Bengio said he disagreed with the regulatory capture argument because a slowdown by leading AI firms would “cost them financially”.

The US president, Donald Trump, has rejected a slowdown, saying he does not want the US to lose its lead over China in the AI race. View image in fullscreenYoshua Bengio, a professor at the University of Montreal, won a Turing award in 2018. Photograph: Andrej Ivanov/AFP/Getty ImagesBengio spoke as the Canadian and German governments announced funding of up to C$300m (£160m) for his non-profit organisation dedicated to creating an “honest AI” that will act as a guardrail against rogue agents – AI tools that carry out sequences of tasks without human intervention.

What happens next

He shared it with Geoffrey Hinton, who later won a Nobel, and Yann LeCun, the former chief AI scientist at Mark Zuckerberg’s Meta. Bengio’s organisation, LawZero, is developing a system called Scientist AI that will act as a guardrail against AI agents showing deceptive or self-preserving behaviour, such as trying to avoid being turned off. LawZero is also funded by the Gates Foundation, the chipmaker Nvidia, and Coefficient Giving, a philanthropic body linked to effective altruism, a movement that believes AI poses an existential threat.