mlsanityData to 14 Oct 2026

Bayesian Influence Functions for Hessian-Free Data Attribution

arXiv 2509.26544 ↗First posted 30 Sep 2025cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Classical influence functions face significant challenges when applied to deep neural networks, primarily due to non-invertible Hessians and high-dimensional parameter spaces. We propose the local Bayesian influence function (BIF), an extension of classical influence functions that replaces Hessian inversion with loss landscape statistics that can be estimated via stochastic-gradient MCMC sampling. This Hessian-free approach captures higher-order interactions among parameters and scales efficiently to neural networks with billions of parameters. We demonstrate state-of-the-art results on predicting retraining experiments.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

30 of 42 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  8. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  9. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  10. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
  11. 2019An Investigation into Neural Net Optimization via Hessian Eigenvalue Densitycited by 292
  12. 2023TRAK: Attributing Model Behavior at Scalecited by 283
18 more
  1. 2018A mathematical theory of semantic development in deep neural networkscited by 186
  2. 2018Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasiscited by 146
  3. 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
  4. 2021Input Similarity from the Neural Network Perspectivecited by 79
  5. 2023The MiniPile Challenge for Data-Efficient Language Modelscited by 75
  6. 2023The Local Learning Coefficient: A Singularity-Aware Complexity Measurecited by 51
  7. 2020Is SGD a Bayesian sampler? Well, almostcited by 50
  8. 2020Deep Learning is Singular, and That's Goodcited by 49
  9. 2024Scalable Influence and Fact Tracing for Large Language Model Pretrainingcited by 42
  10. 2017Covariances, Robustness, and Variational Bayescited by 40
  11. 2024Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficientcited by 40
  12. 2025You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisationcited by 17
  13. 202494% on CIFAR-10 in 3.29 Seconds on a Single GPUcited by 16
  14. 2025Better Training Data Attribution via Better Inverse Hessian-Vector Productscited by 10
  15. 2025From Global to Local: A Scalable Benchmark for Local Posterior Samplingcited by 5
  16. 2023W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimatorscited by 3
  17. 2025Modes of Sequence Models and Learning Coefficientscited by 3
  18. 2023The Bayesian Infinitesimal Jackknife for Variancecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 13.

14 papers.

Sort:

By area:

  1. 2025Influence Dynamics and Stagewise Data Attributioncited by 7
  2. 2026Patterning: The Dual of Interpretabilitycited by 7
  3. 2026Stagewise Reinforcement Learning and the Geometry of the Regret Landscapecited by 4
  4. 2026From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretabilitycited by 4
  5. 2026Towards Worst-Case Guarantees with Scale-Aware Interpretabilitycited by 3
  6. 2026Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learningcited by 2
  7. 2026Interaction-Aware Influence Functions for Group Attributioncited by 2
  8. 2026Mechanistic Anomaly Detection via Functional Attributioncited by 1
  9. 2026Linear Response Estimators for Singular Statistical Modelscited by 1
  10. 2026Interpreting Reinforcement Learning Agents with Susceptibilitiescited by 1
  11. 2026Dead Directions: Geometric Singular Learningcited by 1
  12. 2026Patterning in Practice: Debiasing Reward Models with Susceptibilitiescited by 0
2 more
  1. 2026From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attributioncited by 0
  2. 2026Correct, Don't Delete: Mitigating Emergent Misalignment with Corrective Supervisioncited by 0