mlsanityData to 7 Oct 2026

Revisiting Methods for Finding Influential Examples

arXiv 2111.04683 ↗First posted 8 Nov 2021cs.LG cs.AIcited by 39

Not on the map. Why?

Abstract (arXiv)

Several instance-based explainability methods for finding influential training examples for test-time decisions have been proposed recently, including Influence Functions, TraceIn, Representer Point Selection, Grad-Dot, and Grad-Cos. Typically these methods are evaluated using LOO influence (Cook's distance) as a gold standard, or using various heuristics. In this paper, we show that all of the above methods are unstable, i.e., extremely sensitive to initialization, ordering of the training data, and batch size. We suggest that this is a natural consequence of how in the literature, the influence of examples is assumed to be independent of model state and other examples -- and argue it is not. We show that LOO influence and heuristics are, as a result, poor metrics to measure the quality of instance-based explanations, and instead propose to evaluate such explanations by their ability to detect poisoning attacks. Further, we provide a simple, yet effective baseline to improve all of the above methods and show how it leads to very significant improvements on downstream tasks.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 14 of its 27 references that are in the map. First posted on arXiv 8 Nov 2021.

Builds on

14 of 27 references.

References · 14

  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  2. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  3. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  4. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  5. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  6. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 252
  7. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  8. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  9. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  10. 2019Learning to Deceive with Attention-Based Explanationscited by 111
  11. 2021Input Similarity from the Neural Network Perspectivecited by 78
  12. 2018What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Playcited by 52
2 more
  1. 2020Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencodercited by 27
  2. 2016"Influence Sketching": Finding Influential Samples In Large-Scale Regressionscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 8; 2024: 14; 2025: 10; 2026: 3.

39 papers.

Sort:

By area:

  1. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  2. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
  3. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  4. 2023Foundation Models and Fair Usecited by 112
  5. 2024Data Shapley in One Training Runcited by 76
  6. 2023Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runscited by 52
  7. 2022First is Better Than Last for Language Data Influencecited by 34
  8. 2022ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Datacited by 34
  9. 2024Capturing the Temporal Dependence of Training Data Influencecited by 34
  10. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 33
  11. 2024Rethinking Data Shapley for Data Selection Tasks: Misleads and Meritscited by 29
  12. 2023Theoretical and Practical Perspectives on what Influence Functions Docited by 28
27 more
  1. 2023Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasetscited by 18
  2. 2024Unlearning Traces the Influential Training Data of Language Modelscited by 18
  3. 2024textttdattri: A Library for Efficient Data Attributioncited by 18
  4. 2023A Bayesian Approach To Analysing Training Data Attribution In Deep Learningcited by 11
  5. 2023Unifying Corroborative and Contributive Attributions in Large Language Modelscited by 10
  6. 2025Better Training Data Attribution via Better Inverse Hessian-Vector Productscited by 10
  7. 2024Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspectivecited by 9
  8. 2024On Training Data Influence of GPT Modelscited by 9
  9. 2023Gradient-Based Automated Iterative Recovery for Parameter-Efficient Tuningcited by 5
  10. 2024Efficient Ensembles Improve Training Data Attributioncited by 5
  11. 2025Coresets from Trajectories: Selecting Data via Correlation of Loss Differencescited by 3
  12. 2024Tracing Privacy Leakage of Language Models to Training Data via Adjusted Influence Functionscited by 2
  13. 2024Quanda: An Interpretability Toolkit for Training Data Attribution Evaluation and Beyondcited by 2
  14. 2025Z0-Inf: Zeroth Order Approximation for Data Influencecited by 2
  15. 2023Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Modelscited by 1
  16. 2024Enhancing High-Energy Particle Physics Collision Analysis through Graph Data Attribution Techniquescited by 1
  17. 2024MelissaDL x Breed: Towards Data-Efficient On-line Supervised Training of Multi-parametric Surrogates with Active Learningcited by 1
  18. 2025Finding the Muses: Identifying Coresets through Loss Trajectoriescited by 1
  19. 2025KAIROS: Scalable Model-Agnostic Data Valuationcited by 1
  20. 2025ClinStructor: AI-Powered Structuring of Unstructured Clinical Textscited by 1
  21. 2026Compact Example-Based Explanations for Language Modelscited by 1
  22. 2026Training data attribution in diffusion models via mirrored unlearning and noise-consistent skewcited by 1
  23. 2025Exploring Training Data Attribution under Limited Access Constraintscited by 0
  24. 2025f-INE: A Hypothesis Testing Framework for Estimating Influence under Training Randomnesscited by 0
  25. 2025Additive Large Language Models for Semi-Structured Textcited by 0
  26. 2025Natural Geometry of Robust Data Attribution: From Convex Models to Deep Networkscited by 0
  27. 2026Constraint-Data-Value-Maximization: Utilizing Data Attribution for Effective Data Pruning in Low-Data Environmentscited by 0