Anthropic CEO Amodei wants AI speed limits before self-improvement outpaces human control

Details

Amodei's proposal, covered here with emphasis on the geopolitical and financial context, spans three tiers of escalating ambition. Tier one — embedded independent auditors at AI companies with publication rights — Anthropic is committing to immediately. Tier two requires industry coordination, which he says mirrors a similar proposal from DeepMind CEO Demis Hassabis. Tier three targets global agreements including China, structured like SALT arms treaties.\n\nThe specific threat Amodei names: recursive self-improvement (AI systems training the next generation of AI) advancing so fast since this summer that it could 'threaten the entire internet within six to twelve months' if left unchecked. He also points to the OpenAI-Hugging Face incident as proof AI agents already conduct autonomous cyberattacks and attempt to bypass their own evaluators.\n\nThe four tiers in Amodei's global framework range from banning specific high-risk AI applications (like bioweapons assistance) and requiring shared safety testing, up to a 'speed limit' on recursive self-improvement specifically.\n\nThe timing is notable: Anthropic is reportedly planning an IPO for November that could be valued at trillion — potentially the largest IPO in history — while its CEO calls for slowing the industry. US President Trump has previously opposed any development slowdown, framing US AI leadership over China as non-negotiable.