mlsanityData to 14 Oct 2026

Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution

arXiv 2401.15866 ↗First posted 29 Jan 2024cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Many tasks in explainable machine learning, such as data valuation and feature attribution, perform expensive computation for each data point and are intractable for large datasets. These methods require efficient approximations, and although amortizing the process by learning a network to directly predict the desired output is a promising solution, training such models with exact labels is often infeasible. We therefore explore training amortized models with noisy labels, and we find that this is inexpensive and surprisingly effective. Through theoretical analysis of the label noise and experiments with various models and datasets, we show that this approach tolerates high noise levels and significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 76 of its 112 references that are in the map. First posted on arXiv 29 Jan 2024.

Builds on

76 of 112 references.

References · 76

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  12. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
38 more
  1. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  2. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  3. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  4. 2016HyperNetworkscited by 1,027
  5. 2019Introduction to Online Convex Optimizationcited by 803
  6. 2020Quantifying Attention Flow in Transformerscited by 775
  7. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  8. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  9. 2014Convex Optimization: Algorithms and Complexitycited by 575
  10. 2019Meta-Learning with Implicit Gradientscited by 525
  11. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  12. 2017Interpretation of Neural Networks is Fragilecited by 444
  13. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  14. 2016Top-down Neural Attention by Excitation Backpropcited by 321
  15. 2023TRAK: Attributing Model Behavior at Scalecited by 283
  16. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 281
  17. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
  18. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  19. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  20. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  21. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  22. 2020fastai: A Layered API for Deep Learningcited by 183
  23. 2021Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learningcited by 165
  24. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
  25. 2020Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regressioncited by 154
  26. 2019Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithmscited by 148
  27. 2019Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifierscited by 144
  28. 2019The Shapley Taylor Interaction Indexcited by 143
  29. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 125
  30. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 123
  31. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  32. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 105
  33. 2021Sampling Permutations for Shapley Value Estimationcited by 102
  34. 2019CXPlain: Causal Explanations for Model Interpretation under Uncertaintycited by 98
  35. 2018EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAEcited by 87
  36. 2020A Distributional Framework for Data Valuationcited by 80
  37. 2019Explanation by Progressive Exaggerationcited by 76
  38. 2023LAVA: Data Valuation without Pre-Specified Learning Algorithmscited by 69

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 9; 2026: 7.

21 papers.

Sort:

By area:

  1. 2024Data Shapley in One Training Runcited by 81
  2. 2024Capturing the Temporal Dependence of Training Data Influencecited by 35
  3. 2025Learning to Attribute with Attentioncited by 13
  4. 2025Enhancing Training Data Attribution with Representational Optimizationcited by 8
  5. 2024SHAP zero Explains Biological Sequence Models with Near-zero Marginal Cost for Future Queriescited by 4
  6. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  7. 2026In-Run Data Shapley for Adam Optimizercited by 3
  8. 2024Selective Explanationscited by 2
  9. 2024On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspectivecited by 2
  10. 2025MAnchors: Memorization-Based Acceleration of Anchors via Rule Reuse and Transformationcited by 2
  11. 2025Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Modelscited by 2
  12. 2025Unifying and Optimizing Data Values for Selection via Sequential Decision-Makingcited by 1
9 more
  1. 2025KAIROS: Scalable Model-Agnostic Data Valuationcited by 1
  2. 2025Approximating Shapley Explanations in Reinforcement Learningcited by 1
  3. 2025Optimizing Product Provenance Verification using Data Valuation Methodscited by 0
  4. 2026Learn to Rank: Visual Attribution by Learning Importance Rankingcited by 0
  5. 2026Real-Time Visual Attribution Streaming in Thinking Modelcited by 0
  6. 2026Rethinking XAI Evaluation: A Human-Centered Audit of Shapley Benchmarks in High-Stakes Settingscited by 0
  7. 2026On the Fragility of Data Attribution When Learning Is Distributedcited by 0
  8. 2026Dynamic Shapley Computationcited by 0
  9. 2026Marginal Log-Likelihood Increments under Dirichlet-Smoothed Markov Estimationcited by 0