mlsanityData to 14 Oct 2026

When unlearning is free: leveraging low influence points to reduce computational costs

arXiv 2512.05254 ↗First posted 4 Dec 2025cs.LGcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

As concerns around data privacy in machine learning grow, the ability to unlearn, or remove, specific data points from trained models becomes increasingly important. While state of the art unlearning methods have emerged in response, they typically treat all points in the forget set equally. In this work, we challenge this approach by asking whether points that have a negligible impact on the model's learning need to be removed. Through a comparative analysis of influence functions across language and vision tasks, we identify subsets of training data with negligible impact on model outputs. Leveraging this insight, we propose an efficient unlearning framework that reduces the size of datasets before unlearning leading to significant computational savings (up to approximately 50 percent) on real world empirical examples.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 18 of its 27 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

18 of 27 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning28 Jan
  2. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competition13 Jun
  3. 2023Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutions8 Jul

Other references · 15

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2016Deep Learning with Differential Privacycited by 2,728
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  8. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  9. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  10. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 498
  11. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  12. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
3 more
  1. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  2. 2019CASIA-SURF: A Large-scale Multi-modal Benchmark for Face Anti-spoofingcited by 61
  3. 2020An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference?cited by 27

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mitigating Privacy Risk via Forget Set-Free Unlearning
  2. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning
  3. 2023Learn to Unlearn: A Survey on Machine Unlearning
  4. 2021Machine Unlearning of Features and Labels
  5. 2024Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning
  6. 2023Machine Unlearning: Solutions and Challenges
  7. 2025Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective
  8. 2024Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearning