Accumulative SGD Influence Estimation for Data Attribution
Not on the map. Why?
Abstract (arXiv)
Modern data-centric AI needs precise per-sample influence. Standard SGD-IE approximates leave-one-out effects by summing per-epoch surrogates and ignores cross-epoch compounding, which misranks critical examples. We propose ACC-SGD-IE, a trajectory-aware estimator that propagates the leave-one-out perturbation across training and updates an accumulative influence state at each step. In smooth strongly convex settings it achieves geometric error contraction and, in smooth non-convex regimes, it tightens error bounds; larger mini-batches further reduce constants. Empirically, on Adult, 20 Newsgroups, and MNIST under clean and corrupted data and both convex and non-convex training, ACC-SGD-IE yields more accurate influence estimates, especially over long epochs. For downstream data cleansing it more reliably flags noisy samples, producing models trained on ACC-SGD-IE cleaned data that outperform those cleaned with SGD-IE.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 30 of its 57 references that are in the map. First posted on arXiv 30 Oct 2025.
Builds on
30 of 57 references.
References · 30
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 678
- 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
- 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 533
- 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
- 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 528
- 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
- 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 425
- 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 339
- 2023Studying Large Language Model Generalization with Influence Functionscited by 326
- 2023TRAK: Attributing Model Behavior at Scalecited by 283
- 2020Influence Functions in Deep Learning Are Fragilecited by 252
18 more
- 2023Data-centric Artificial Intelligence: A Surveycited by 203
- 2022If Influence Functions are the Answer, Then What is the Question?cited by 144
- 2022Training Data Influence Analysis and Estimation: A Surveycited by 135
- 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 133
- 2022Coverage-centric Coreset Selection for High Pruning Ratescited by 117
- 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 105
- 2024Data-efficient Fine-tuning for LLM-based Recommendationcited by 104
- 2019Data Cleansing for Models Trained with SGDcited by 67
- 2020Influence Function based Data Poisoning Attacks to Top-N Recommender Systemscited by 63
- 2020Introduction to Core-sets: an Updated Surveycited by 45
- 2022Exploring Example Influence in Continual Learningcited by 44
- 2025PersonaX: A Recommendation Agent Oriented User Modeling Framework for Long Behavior Sequencecited by 33
- 2021Interpretable Data-Based Explanations for Fairness Debuggingcited by 30
- 2022Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernelscited by 25
- 2020Toward Optimal Probabilistic Active Learning Using a Bayesian Approachcited by 6
- 2024lpNTK: Better Generalisation with Less Data via Sample Interaction During Learningcited by 6
- 2024Data Efficiency for Large Recommendation Modelscited by 2
- 2024Training-free Heterogeneous Graph Condensation via Data Selectioncited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.