mlsanityData to 7 Oct 2026

Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments

arXiv 2206.10013 ↗First posted 20 Jun 2022cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

We develop a new, principled algorithm for estimating the contribution of training data points to the behavior of a deep learning model, such as a specific prediction it makes. Our algorithm estimates the AME, a quantity that measures the expected (average) marginal effect of adding a data point to a subset of the training data, sampled from a given distribution. When subsets are sampled from the uniform distribution, the AME reduces to the well-known Shapley value. Our approach is inspired by causal inference and randomized experiments: we sample different subsets of the training data to train multiple submodels, and evaluate each submodel's behavior. We then use a LASSO regression to jointly estimate the AME of each data point, based on the subset compositions. Under sparsity assumptions (k ll N datapoints have large AME), our estimator requires only O(klog N) randomized submodel trainings, improving upon the best prior Shapley value estimators.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 39 of its 92 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

39 of 92 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  6. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  7. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  8. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  9. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  10. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  11. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  12. 2018Spectral Signatures in Backdoor Attackscited by 457
27 more
  1. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  2. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  3. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 279
  4. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  5. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  6. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  7. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  8. 2017A Rewriting System for Convex Optimization Problemscited by 178
  9. 2021Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learningcited by 162
  10. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  11. 2020Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regressioncited by 153
  12. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 146
  13. 2018Stronger Data Poisoning Attacks Break Data Sanitization Defensescited by 125
  14. 2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 123
  15. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 123
  16. 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 121
  17. 2016Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selectioncited by 120
  18. 2019Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainabilitycited by 114
  19. 2021Sampling Permutations for Shapley Value Estimationcited by 101
  20. 2020A Distributional Framework for Data Valuationcited by 79
  21. 2020Efficient nonparametric statistical inference on population feature importance using Shapley valuescited by 61
  22. 2013Bounding the Estimation Error of Sampling-based Shapley Value Approximationcited by 56
  23. 2019Model Agnostic Defence against Backdoor Attacks in Machine Learningcited by 52
  24. 2017BoostClean: Automated Error Detection and Repair for Machine Learningcited by 23
  25. 2020Complaint-driven Training Data Debugging for Query 2.0cited by 16
  26. 2021Joint Shapley values: a measure of joint feature importancecited by 15
  27. 2016Debugging Machine Learning Taskscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 13; 2024: 22; 2025: 8; 2026: 5.

50 papers.

Sort:

By area:

  1. 2023TRAK: Attributing Model Behavior at Scalecited by 273
  2. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  3. 2024Data Shapley in One Training Runcited by 76
  4. 2023OpenDataVal: a Unified Benchmark for Data Valuationcited by 55
  5. 2023Intriguing Properties of Data Attribution on Diffusion Modelscited by 47
  6. 2023The Journey, Not the Destination: How Data Guides Diffusion Modelscited by 42
  7. 2023Data-OOB: Out-of-bag Estimate as a Simple and Efficient Data Valuecited by 38
  8. 2024Rethinking Data Shapley for Data Selection Tasks: Misleads and Meritscited by 29
  9. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  10. 2025DataMIL: Selecting Data for Robot Imitation Learning with Datamodelscited by 25
  11. 2024Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Modelscited by 21
  12. 2023On the Variance of Neural Network Training with respect to Test Sets and Distributionscited by 20
38 more
  1. 2024Causal Estimation of Memorisation Profilescited by 20
  2. 2023Threshold KNN-Shapley: A Linear-Time and Privacy-Friendly Approach to Data Valuationcited by 19
  3. 2025Regression-adjusted Monte Carlo Estimators for Shapley Values and Probabilistic Valuescited by 19
  4. 2024Efficient Data Shapley for Weighted Nearest Neighbor Algorithmscited by 18
  5. 2024An Economic Solution to Copyright Challenges of Generative AIcited by 18
  6. 2023Data Valuation and Detections in Federated Learningcited by 17
  7. 2023Rethinking Backdoor Attackscited by 14
  8. 2024Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimationcited by 13
  9. 2023A Note on "Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms"cited by 11
  10. 2023Selectivity Drives Productivity: Efficient Dataset Pruning for Enhanced Transfer Learningcited by 11
  11. 2022XInsight: eXplainable Data Analysis Through The Lens of Causalitycited by 9
  12. 2024Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspectivecited by 9
  13. 2024One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficientlycited by 9
  14. 20232D-Shapley: A Framework for Fragmented Data Valuationcited by 7
  15. 2025Fast Data Attribution for Text-to-Image Modelscited by 7
  16. 2023Data Optimization in Deep Learning: A Surveycited by 6
  17. 2024Sparse, Efficient and Explainable Data Attribution with DualXDAcited by 6
  18. 2024Task-Oriented GNNs Training on Large Knowledge Graphs for Accurate and Efficient Modelingcited by 5
  19. 2024Improve Knowledge Distillation via Label Revision and Data Selectioncited by 5
  20. 2024Data Debiasing with Datamodels (D3M): Improving Subgroup Robustness via Data Selectioncited by 5
  21. 2024Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methodscited by 4
  22. 2024CHG Shapley: Efficient Data Valuation and Selection towards Trustworthy Machine Learningcited by 3
  23. 2026In-Run Data Shapley for Adam Optimizercited by 3
  24. 2024Uncertainty-aware Evaluation of Auxiliary Anomalies with the Expected Anomaly Posteriorcited by 2
  25. 2024Adversarial Attacks on Data Attributioncited by 2
  26. 2024Is Data Valuation Learnable and Interpretable?cited by 1
  27. 2024LossVal: Efficient Data Valuation for Neural Networkscited by 1
  28. 2025Towards Unified Attribution in Explainable AI, Data-Centric AI, and Mechanistic Interpretabilitycited by 1
  29. 2025Unifying and Optimizing Data Values for Selection via Sequential Decision-Makingcited by 1
  30. 2026Local Shapley: Model-Induced Locality and Optimal Reuse in Data Valuationcited by 1
  31. 2026Adalina: Adaptive Linear Approximation for the Shapley Value and Beyondcited by 1
  32. 2024Neural Dynamic Data Valuation via Stochastic State-Adjoint Trajectoriescited by 0
  33. 2024Data Valuation by Fusing Global and Local Statistical Informationcited by 0
  34. 2025Data Overvaluation Attack and Truthful Data Valuation in Federated Learningcited by 0
  35. 2025A Causal Framework for Aligning Image Quality Metrics and Deep Neural Network Robustnesscited by 0
  36. 2025Onto-Epistemological Analysis of AI Explanationscited by 0
  37. 2026Feature Attribution in Directed Acyclic Graphs Using Edge Interventioncited by 0
  38. 2026Localizing Global Discrepancies: Marginal Contributions and Contextual Anomaly Detectioncited by 0