mlsanityData to 14 Oct 2026

Towards Robust Influence Functions with Flat Validation Minima

arXiv 2505.19097 ↗First posted 25 May 2025cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

The Influence Function (IF) is a widely used technique for assessing the impact of individual training samples on model predictions. However, existing IF methods often fail to provide reliable influence estimates in deep neural networks, particularly when applied to noisy training data. This issue does not stem from inaccuracies in parameter change estimation, which has been the primary focus of prior research, but rather from deficiencies in loss change estimation, specifically due to the sharpness of validation risk. In this work, we establish a theoretical connection between influence estimation error, validation set risk, and its sharpness, underscoring the importance of flat validation minima for accurate influence estimation. Furthermore, we introduce a novel estimation form of Influence Function specifically designed for flat validation minima. Experimental results across various tasks validate the superiority of our approach.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 25 May 2025.

Builds on

28 of 40 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  5. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  7. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  8. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  9. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  10. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
  11. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  12. 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 261
16 more
  1. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 256
  2. 2020Influence Functions in Deep Learning Are Fragilecited by 252
  3. 2022Surrogate Gap Minimization Improves Sharpness-Aware Trainingcited by 172
  4. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  5. 2022If Influence Functions are the Answer, Then What is the Question?cited by 144
  6. 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
  7. 2021Scaling Up Influence Functionscited by 132
  8. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 123
  9. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  10. 2019Image Classification with Deep Learning in the Presence of Noisy Labels: A Surveycited by 101
  11. 2022Fisher SAM: Information Geometry and Sharpness Aware Minimisationcited by 84
  12. 2019Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysiscited by 59
  13. 2024Friendly Sharpness-Aware Minimizationcited by 53
  14. 2023An Adaptive Policy to Employ Sharpness-Aware Minimizationcited by 43
  15. 2022Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscapecited by 39
  16. 2023Enhancing Sharpness-Aware Optimization Through Variance Suppressioncited by 37

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Better Hessians Matter: Studying the Impact of Curvature Approximations in Influence Functionscited by 3
  2. 2026Interaction-Aware Influence Functions for Group Attributioncited by 2
  3. 2025CHIPS: Efficient CLIP Adaptation via Curvature-aware Hybrid Influence-based Data Selectioncited by 1
  4. 2026Distill on a Diet: Efficient Knowledge Distillation via Learnable Data Pruningcited by 1