What’s behind the AI industry’s latest warnings of doom?
| Source: TechCrunch AI
Tags: Anthropic, AI safety, existential risk, Jacob Coxon, IPO, alignment, Dario Amodei
TechCrunch's Equity podcast unpacks the wave of existential risk warnings from Anthropic insiders: a researcher's resignation citing AI danger, and Anthropic's alignment lead claiming >10% odds of human extinction within a decade — and whether these warnings reflect genuine safety concern or serve as IPO positioning ahead of Anthropic's S-1 filing.
Details
The AI safety debate intensified after Jacob Coxon, a researcher who had worked at both OpenAI and Anthropic, resigned from Anthropic claiming frontier AI labs are 'gambling with our lives.' Anthropic's alignment lead then amplified Coxon's post publicly, stating 'We really do earnestly believe AI could kill all humans!' and personally estimating a >10% probability of AI killing humans within the next decade. TechCrunch's Equity hosts debate whether these warnings represent genuine safety advocacy or serve secondary IPO positioning — with Anthropic's S-1 expected soon, the company's legal team may need to formally disclose existential risk as a material business risk. The timing coincided with a Hugging Face breach involving an OpenAI internal model, compounding existing anxieties about AI security. The panel raises valid skepticism: the >10% figure is unmoored from any clear methodology, and the 'we' in the alignment lead's post conflates personal belief with company position. Anthropic CEO Dario Amodei later published a plan for more cautious AI development, though this episode was recorded before that response. The underlying debate — about genuine existential risk versus risk as a PR and regulatory lever — will likely intensify as frontier AI labs approach public markets.