When Unlearning Is Free: Leveraging Low Influence Points to Reduce Computational Costs

Details

A team from Apple, Harvard, and UCL has published a machine unlearning framework that challenges the standard approach of treating all data in a forget set as equally necessary to remove. Published through Apple ML Research in July 2026, the paper shows that a meaningful share of training data in any forget set has negligible influence on model outputs — and that skipping these points during unlearning saves up to roughly 50% of compute on real-world empirical examples. The method uses influence functions to estimate how much each training point shaped a model's learned parameters. After running this analysis across both language and vision tasks, the researchers identify low-influence subsets that can be excluded from the unlearning procedure without meaningfully changing the outcome. The practical use case is privacy compliance. Regulations like GDPR create a right-to-be-forgotten requirement that technically demands removing specific training data's effect from deployed models. Current unlearning methods are expensive enough to make frequent compliance at scale operationally difficult. A 50% reduction in unlearning compute would change that calculus, especially for large language models trained on broad datasets where removal requests may arrive regularly. The work is at the research stage: the efficiency gains come from empirical examples and formal analysis, not a production deployment. Authors include Anat Kleiman (Harvard/Apple), Robert Fisher, Ben Deaner (UCL), Udi Wieder, and Vitaly Feldman.