mlsanityData to 7 Oct 2026

A Simple and Efficient Baseline for Data Attribution on Images

arXiv 2311.03386 ↗First posted 3 Nov 2023cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Data attribution methods play a crucial role in understanding machine learning models, providing insight into which training data points are most responsible for model outputs during deployment. However, current state-of-the-art approaches require a large ensemble of as many as 300,000 models to accurately attribute model predictions. These approaches therefore come at a high computational cost, are memory intensive, and are hard to scale to large models or datasets. In this work, we focus on a minimalist baseline, utilizing the feature space of a backbone pretrained via self-supervised learning to perform data attribution. Our method is model-agnostic and scales easily to large datasets. We show results on CIFAR-10 and ImageNet, achieving strong performance that rivals or outperforms state-of-the-art approaches at a fraction of the compute or memory cost. Contrary to prior work, our results reinforce the intuition that a model's prediction on one image is most impacted by visually similar training samples. Our approach serves as a simple and efficient baseline for data attribution on images.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

31 of 42 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  11. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  12. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
19 more
  1. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  2. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  3. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  4. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  5. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  6. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  7. 2023Studying Large Language Model Generalization with Influence Functionscited by 311
  8. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 279
  9. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  10. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  11. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  12. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  13. 2021Influence Selection for Active Learningcited by 82
  14. 2021Input Similarity from the Neural Network Perspectivecited by 78
  15. 2023FFCV: Accelerating Training by Removing Data Bottleneckscited by 49
  16. 2019Model Similarity Mitigates Test Set Overusecited by 37
  17. 2021Leave-one-out Unfairnesscited by 36
  18. 2022Identifying a Training-Set Attack's Target Using Renormalized Influence Estimationcited by 32
  19. 2022ModelDiff: A Framework for Comparing Learning Algorithmscited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 33
  2. 2025Learning to Weight Parameters for Training Data Attributioncited by 3
  3. 2024Quanda: An Interpretability Toolkit for Training Data Attribution Evaluation and Beyondcited by 2
  4. 2026A Bayesian Information-Theoretic Approach to Data Attributioncited by 1
  5. 2026Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behaviorcited by 0