AIBig AI Wants to Slow Down AI Research. Is It a Safety Pause or a Strategic Retreat?
Any successful effort at “pacing the rate of capabilities advancement so that risk prevention has time to keep up”, as Anthropic chief executive Dario Amodei puts it, will require significant cooperation between rival companies – and nations.
Over the weekend, Anthropic chief executive Dario Amodei called for artificial intelligence (AI) companies, including his own, to slow down their work. Sam Altman and Elon Musk, heads of rivals OpenAI and xAI respectively, agreed.
The move reflects concern across the AI industry and more broadly about the dangers of new, rapidly improving systems. Recent high-profile incidents such as OpenAI AI agents hacking another company and hijacking a public website have shown current systems can break out of safety confines – and even more capable systems are in development.
Further complicating AI safety is the tension between safety and performance. Companies will be reluctant to limit the performance of their models in the name of safety for fear of losing ground to competitors. US President Donald Trump has also rejected calls for a slowdown, for fear of losing ground to China.
This means any successful effort at “pacing the rate of capabilities advancement so that risk prevention has time to keep up”, as Amodei puts it, will require significant cooperation between rival companies – and nations.
Risk Minimization
New technologies often bring new risks, and new concerns. Often governments, researchers and companies do find ways to manage those risks.
In 1975, the Asilomar conference on then-new DNA technologies did much to ensure research didn’t get ahead of our understanding of safety and risk. Similarly, in the 1990s, the US government attempted to build a consensus on limiting cryptography and computer security.
The AI situation has an extra twist: the industry is in the middle of a gold rush. Scientific rivals may be able to restrain themselves, but commercial rivals rarely hold back.
There are clear precedents for self-regulation failing in the face of competitive pressure. In the Boeing 737 Max disaster in 2018, for example, pressure to catch up to rival Airbus led Boeing to hide the limitations of the Max, which ultimately cost lives.
In the AI race, the scale of the competitive tension is even greater. Anthropic and OpenAI are both competing to establish market dominance before pursuing share market listings that could raise tens or hundreds of billions of dollars.
And at the nation-state level, the stakes are higher again, with the US and China each hoping to use the new technology for geopolitical advantage.
The Politics of a Pause
Amodei’s slowdown proposal centers around a three-step plan: embedding independent third-party safety reviewers, establishing coordinated industry safety standards within democratic nations, and eventually securing global agreements. This would include strict limits on AI chip exports to companies
