mlsanityData to 7 Oct 2026

The Approximate Fisher Influence Function: Faster Estimation of Data Influence in Statistical Models

arXiv 2407.08169 ↗First posted 11 Jul 2024cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Quantifying the influence of infinitesimal changes in training data on model performance is crucial for understanding and improving machine learning models. In this work, we reformulate this problem as a weighted empirical risk minimization and enhance existing influence function-based methods by using information geometry to derive a new algorithm to estimate influence. Our formulation proves versatile across various applications, and we further demonstrate in simulations how it remains informative even in non-convex cases. Furthermore, we show that our method offers significant computational advantages over current Newton step-based methods.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

28 of 53 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Decoupled Weight Decay Regularizationcited by 1,722
  4. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  5. 2014New insights and perspectives on the natural gradient methodcited by 435
  6. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
  7. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  8. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  9. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 213
  10. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
  11. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  12. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
16 more
  1. 2021Scaling Up Influence Functionscited by 130
  2. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  3. 2020Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networkscited by 96
  4. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 88
  5. 2023Machine Unlearning: Solutions and Challengescited by 85
  6. 2019LassoNet: A Neural Network with Feature Sparsitycited by 81
  7. 2019On Second-Order Group Influence Functions for Black-Box Predictionscited by 78
  8. 2018A Swiss Army Infinitesimal Jackknifecited by 51
  9. 2022Algorithms that Approximate Data Removal: New Results and Limitationscited by 39
  10. 2021Selective Regression Under Fairness Criteriacited by 34
  11. 2020An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference?cited by 27
  12. 2022Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refittingcited by 26
  13. 2020Approximate Cross-validation: Guarantees for Model Assessment and Selectioncited by 14
  14. 2019A Higher-Order Swiss Army Infinitesimal Jackknifecited by 13
  15. 2017On Optimal Generalizability in Parametric Learningcited by 12
  16. 2024A direct proof of a unified law of robustness for Bregman divergence lossescited by 4

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Exploring Incremental Unlearning: Techniques, Challenges, and Future Directionscited by 2
  2. 2025On the Accuracy of Newton Step and Influence Function Data Attributionscited by 0