Anthropic calls for a global pause on advanced AI development, citing risks of recursive self-improvement
The US AI company Anthropic has proposed a global coordination mechanism to slow or suspend the development of frontier AI systems, arguing that society needs time to adapt to the technology's rapid progress.
The proposal
Anthropic, the San Francisco-based creator of the Claude chatbot, published a text through its Anthropic Institute think tank on Thursday, 4 June, advocating for a worldwide slowdown or temporary suspension of advanced AI development. The company argued that such a pause would allow societal structures and alignment research to catch up with technological progress.
We believe it would be good for the world to have the ability to slow down or temporarily suspend the development of cutting-edge AI, in order to allow societal structures and alignment research to keep pace with the technology's progress.
The company stressed that any pause must be coordinated globally. If a single firm halted work unilaterally, competitors would simply overtake it. Anthropic named OpenAI, Google, and China's DeepSeek among its fierce rivals.
The risk of recursive self-improvement
Anthropic justified its proposal by unveiling internal data illustrating the possibility of one day reaching "recursive self-improvement" — the point where AI systems become capable of training their own successors, reducing human intervention. The danger, the company suggested, could arise from this self-generation spiralling out of control or diverging from humanity's interests.
"If current trends continue," the company wrote, such a scenario becomes "plausible," though it added that there is no guarantee it is on the horizon.
A nuclear treaty precedent
To make its case, Anthropic cited the precedent of the Intermediate-Range Nuclear Forces (INF) Treaty. It noted that this framework took decades to establish, but that time is more pressing for AI, which it described as "far easier to conceal than missile silos."
In the absence of a global coordination mechanism, companies and governments will have to make difficult safety decisions, under pressure from competition and geopolitical stakes.
Political and legal headwinds
The proposal is expected to face strong resistance in the United States, where many officials and tech leaders reject any slowdown that might give China a decisive advantage. Donald Trump, however, stated that he had discussed the possibility of cooperation with China on AI safety during a recent visit to Beijing.
Anthropic is currently in a legal dispute with the US Department of Defense over the unrestricted military use of its technology, which the company has refused. The Pentagon subsequently classified Anthropic as a "supply chain security risk," a designation the company is challenging in court. The firm has also restricted the release of Mythos, its most advanced model, to establish cybersecurity fixes. On Monday, Anthropic filed for its initial public offering.
Industry scepticism
Founded by OpenAI dissidents, Anthropic has cultivated an image of a lab centred on ethics and safety. It faces criticism from industry figures and White House officials who accuse it of exaggerating risks, with some calling its approach a "fear marketing" strategy amid a race with OpenAI to go public.
