mlsanityData to 14 Oct 2026

Estimating Training Data Influence by Tracing Gradient Descent

arXiv 2002.08484 ↗First posted 19 Feb 2020cs.LG stat.MLcited by 580

Not on the map. Why?

Cited by 35 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a method called TracIn that computes the influence of a training example on a prediction made by the model. The idea is to trace how the loss on the test point changes during the training process whenever the training example of interest was utilized. We provide a scalable implementation of TracIn via: (a) a first-order gradient approximation to the exact computation, (b) saved checkpoints of standard training procedures, and (c) cherry-picking layers of a deep neural network. In contrast with previously proposed methods, TracIn is simple to implement; all it needs is the ability to work with gradients, checkpoints, and loss functions. The method is general. It applies to any machine learning model trained using stochastic gradient descent or a variant of it, agnostic of architecture, domain and task. We expect the method to be widely useful within processes that study and improve training data.

Key facts. Cited by 580 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 45; 2023: 68; 2024: 116; 2025: 160; 2026: 168.

580 papers.

Sort:

By area:

  1. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  2. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  3. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 533
  4. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  5. 2023Explainability for Large Language Models: A Surveycited by 333
  6. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  7. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  8. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  9. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  10. 2022If Influence Functions are the Answer, Then What is the Question?cited by 144
  11. 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
  12. 2021Scaling Up Influence Functionscited by 132
38 more
  1. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 123
  2. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 118
  3. 2024Learning Dynamics of LLM Finetuningcited by 95
  4. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 94
  5. 2023DoGE: Domain Reweighting with Generalization Estimationcited by 90
  6. 2024β-DPO: Direct Preference Optimization with Dynamic βcited by 90
  7. 2024Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Datacited by 85
  8. 2024Data Shapley in One Training Runcited by 81
  9. 2023Fast Model Debias with Machine Unlearningcited by 72
  10. 2023LAVA: Data Valuation without Pre-Specified Learning Algorithmscited by 69
  11. 2023In-context Example Selection with Influencescited by 61
  12. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  13. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  14. 2022Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlationcited by 59
  15. 2022Data Curation Alone Can Stabilize In-context Learningcited by 59
  16. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 58
  17. 2022Concept Activation Regions: A Generalized Framework For Concept-Based Explanationscited by 57
  18. 2023Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runscited by 53
  19. 2022Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experimentscited by 50
  20. 2023Intriguing Properties of Data Attribution on Diffusion Modelscited by 47
  21. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 46
  22. 2023Evaluating Data Attribution for Text-to-Image Modelscited by 45
  23. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Modelscited by 45
  24. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  25. 2023The Journey, Not the Destination: How Data Guides Diffusion Modelscited by 43
  26. 2024Scalable Influence and Fact Tracing for Large Language Model Pretrainingcited by 42
  27. 2023Understanding In-Context Learning via Supportive Pretraining Datacited by 41
  28. 2025Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoningcited by 40
  29. 2021Revisiting Methods for Finding Influential Examplescited by 39
  30. 2023Learning Support and Trivial Prototypes for Interpretable Image Classificationcited by 38
  31. 2023Self-Influence Guided Data Reweighting for Language Model Pre-trainingcited by 37
  32. 2025Emergent Abilities in Large Language Models: A Surveycited by 37
  33. 2023Data Valuation Without Training of a Modelcited by 35
  34. 2022First is Better Than Last for Language Data Influencecited by 34
  35. 2022ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Datacited by 34
  36. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 34
  37. 2024Harnessing Diversity for Important Data Selection in Pretraining Large Language Modelscited by 34
  38. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34

50 most cited of 580 All 580 (JSON)