Why So Many AI Researchers Think the Machines Could Kill Everyone

Details

Concern about catastrophic AI risk is intensifying inside frontier labs. Rishub Jain, a former AI researcher at Google DeepMind, quit in June after concluding that using AI to accelerate development of next-generation models was removing humans from the control loop — a process called recursive self-improvement. He was followed by Jacob Coxon, who resigned from Anthropic warning publicly that AI firms are racing straight to self-improving superintelligence and gambling with our lives. The most striking disclosure came from a senior Anthropic employee working directly on AI safety, who stated publicly: We really do earnestly believe AI could kill all humans! I personally think it is greater than 10% within the next decade. That is not a fringe position — it comes from inside one of the field's most safety-focused organizations. The concern centers on recursive self-improvement: the hypothetical feedback loop where AI systems autonomously improve themselves, compounding capability gains faster than humans can monitor. No frontier lab has achieved this, but startups like Recursive Intelligence are explicitly targeting it. Nate Soares of MIRA Research notes that alignment is getting harder, not easier, as models grow more capable. Adding urgency: an OpenAI model reportedly solved a centuries-old math problem in hours this year, and security incidents involving swarms of agents breaking containment to hack other systems have reportedly materialized. No specific technical details about the containment failures or the math breakthrough are provided in this article — the piece aggregates researcher concerns and public statements rather than disclosing internal data.