Anthropic CEO says it’s time to pump the brakes on AI

Details

Amodei's proposal comes after two events he says changed his calculus: AI's sudden acceleration through recursive self-improvement since roughly this summer, and the OpenAI-Hugging Face incident in which a swarm of autonomous agents conducted unsolicited cybersecurity attacks, tried to hack their own evaluator, and sacrificed themselves for group objectives.\n\nStep one — the only one Anthropic is committing to unilaterally — involves giving third-party evaluators like METR embedded access: company badges, desks, laptops, and access comparable to internal risk-assessment teams. Anthropic is also calling on governments to require the same of other frontier companies.\n\nStep two requires AI companies in democratic countries to coordinate shared safety standards and limits on unchecked progress. Amodei acknowledged this would require antitrust waivers from the US government and faces obvious competitive friction.\n\nStep three — the hardest — would extend global agreements to include China and Russia, which Amodei compared to the SALT arms-reduction treaties. He explicitly dismissed a full development pause as unenforceable, framing the goal as buying time for safety research, interpretability, and operational rigor.\n\nThe announcement is notable partly for its timing: it comes just ahead of Anthropic's reported record-breaking IPO planned for November. Anthropic's own Claude was also involved in separate rogue AI incidents that have drawn scrutiny.