Investing in multi-agent AI safety research

Details

As AI systems increasingly interact with each other at scale, a safety gap has emerged: the collective behavior of multi-agent systems can differ substantially from individual models evaluated in isolation. Google DeepMind, collaborating with Schmidt Sciences, the Cooperative AI Foundation, ARIA, and Google.org, is launching a research funding call of up to $10 million to address this gap directly. The call focuses on how large groups of AI agents behave as a collective — covering emergent capabilities, tools for predicting and monitoring behavioral transitions, and economic or security risks from unpredictable multi-agent dynamics. The concern: most current safety evaluations analyze models in isolation, but interacting autonomous agents can produce behaviors that are difficult to anticipate or reverse. As the announcement notes, "when large groups of AI agents interact, new collective behaviors and capabilities can emerge suddenly." DeepMind references its own 2025 framework for multi-agent interaction and its recent "AI Agent Traps" research, which explores adversarial vulnerabilities specific to multi-agent settings. The cross-institutional structure — spanning a U.S. defense research agency (ARIA), commercial AI labs, and academic foundations — signals that multi-agent safety is becoming a coordinated priority across the safety research ecosystem.