mlsanityData to 7 Oct 2026

Understanding Influence Functions and Datamodels via Harmonic Analysis

arXiv 2210.01072 ↗First posted 3 Oct 2022cs.LG cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

Influence functions estimate effect of individual data points on predictions of the model on test data and were adapted to deep learning in Koh and Liang [2017]. They have been used for detecting data poisoning, detecting helpful and harmful examples, influence of groups of datapoints, etc. Recently, Ilyas et al. [2022] introduced a linear regression method they termed datamodels to predict the effect of training points on outputs on test data. The current paper seeks to provide a better theoretical understanding of such interesting empirical phenomena. The primary tool is harmonic analysis and the idea of noise stability. Contributions include: (a) Exact characterization of the learnt datamodel in terms of Fourier coefficients. (b) An efficient method to estimate the residual error and quality of the optimum linear datamodel without having to train the datamodel. (c) New insights into when influences of groups of datapoints may or may not add up linearly.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 3 Oct 2022.

Builds on

26 of 40 references.

References · 26

  1. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  2. 2018On the Spectral Bias of Neural Networkscited by 1,321
  3. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
  4. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  5. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  6. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  7. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  8. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 279
  9. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  10. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  11. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  12. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
14 more
  1. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
  2. 2021Scaling Up Influence Functionscited by 130
  3. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  4. 2018Understanding the Origins of Bias in Word Embeddingscited by 90
  5. 2019On Second-Order Group Influence Functions for Black-Box Predictionscited by 78
  6. 2021Input Similarity from the Neural Network Perspectivecited by 78
  7. 2019Detecting Adversarial Samples Using Influence Functions and Nearest Neighborscited by 72
  8. 2018Interpreting Black Box Predictions using Fisher Kernelscited by 48
  9. 2019Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributionscited by 48
  10. 2021Interactive Label Cleaning with Example-based Explanationscited by 48
  11. 2020Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functionscited by 34
  12. 2019Can You Trust This Prediction? Auditing Pointwise Reliability After Learningcited by 28
  13. 2020RelatIF: Identifying Explanatory Training Examples via Relative Influencecited by 28
  14. 2021Rethinking Influence Functions of Neural Networks in the Over-parameterized Regimecited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 8; 2025: 6; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  2. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  3. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 103
  4. 2024Data Shapley in One Training Runcited by 76
  5. 2024Most Influential Subset Selection: Challenges, Promises, and Beyondcited by 31
  6. 2024Rethinking Data Shapley for Data Selection Tasks: Misleads and Meritscited by 29
  7. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  8. 2024Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attributioncited by 21
  9. 2023Identification of Negative Transfers in Multitask Learning Using Surrogate Modelscited by 20
  10. 2025MAGIC: Near-Optimal Data Attribution for Deep Learningcited by 20
  11. 2023Natural Example-Based Explainability: a Surveycited by 13
  12. 2025Group-Level Data Selection for Efficient Pretrainingcited by 12
11 more
  1. 2025Better Training Data Attribution via Better Inverse Hessian-Vector Productscited by 10
  2. 2026Contextual Rollout Bandits for Reinforcement Learning with Verifiable Rewardscited by 10
  3. 2024One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficientlycited by 9
  4. 2025DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Taskscited by 6
  5. 2023GMValuator: Similarity-based Data Valuation for Generative Modelscited by 5
  6. 2023Learning to Rank for Active Learning via Multi-Task Bilevel Optimizationcited by 3
  7. 2026How to sketch a learning algorithmcited by 2
  8. 2022Learning to Counterfactually Explain Recommendationscited by 1
  9. 2025Small-to-Large Generalization: Data Influences Models Consistently Across Scalecited by 1
  10. 2025Efficiently Verifiable Proofs of Data Attributioncited by 1
  11. 2026Interaction-Aware Influence Functions for Group Attributioncited by 1