New framework for auditing machine unlearning

Details

Machine unlearning — making an AI model forget specific training data without full retraining — is increasingly required under GDPR's Right to be Forgotten. But verifying that unlearning actually occurred is hard: auditors typically lack access to model internals and must verify through output queries alone.\n\nGoogle's new framework, presented at AISTATS 2026, introduces Regularized f-Divergence Kernel Tests for this verification task. The core problem with existing tools: standard two-sample tests like maximum mean discrepancy (MMD) lose statistical power as models grow larger, requiring impractically large sample sizes to detect whether unlearning genuinely occurred or just appears to have.\n\nThe new framework is provably more sensitive for complex, high-dimensional distributions — it theoretically controls false positive rates for any sample size, and false negative risk converges to zero as sample count increases. It supports different 'distance' notions between output distributions, making it more flexible than MMD for varied compliance scenarios.\n\nThis is directly relevant for any organization deploying AI systems subject to data deletion requests. The practical gap — computational cost of running reliable unlearning audits — is a real blocker for compliance teams. Peer review at a top-tier ML venue (AISTATS) adds credibility beyond a typical corporate blog post.