# Accumulative SGD Influence Estimation for Data Attribution

arXiv: https://arxiv.org/abs/2510.26185 · HTML: https://mlsanity.com/paper/2510.26185/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 30 Oct 2025.

## Builds on (most cited first, 30 of 30)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [Deep Learning on a Data Diet: Finding Important Examples Early in Training](https://mlsanity.com/paper/2107.07075/) (2021)
- [LESS: Selecting Influential Data for Targeted Instruction Tuning](https://mlsanity.com/paper/2402.04333/) (2024)
- [Beyond neural scaling laws: beating power law scaling via data pruning](https://mlsanity.com/paper/2206.14486/) (2022)
- [What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation](https://mlsanity.com/paper/2008.03703/) (2020)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [Selection via Proxy: Efficient Data Selection for Deep Learning](https://mlsanity.com/paper/1906.11829/) (2019)
- [Studying Large Language Model Generalization with Influence Functions](https://mlsanity.com/paper/2308.03296/) (2023)
- [TRAK: Attributing Model Behavior at Scale](https://mlsanity.com/paper/2303.14186/) (2023)
- [Influence Functions in Deep Learning Are Fragile](https://mlsanity.com/paper/2006.14651/) (2020)
- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [If Influence Functions are the Answer, Then What is the Question?](https://mlsanity.com/paper/2209.05364/) (2022)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [Dataset Pruning: Reducing Training Data by Examining Generalization Influence](https://mlsanity.com/paper/2205.09329/) (2022)
- [Coverage-centric Coreset Selection for High Pruning Rates](https://mlsanity.com/paper/2210.15809/) (2022)
- [DsDm: Model-Aware Dataset Selection with Datamodels](https://mlsanity.com/paper/2401.12926/) (2024)
- [Data-efficient Fine-tuning for LLM-based Recommendation](https://mlsanity.com/paper/2401.17197/) (2024)
- [Data Cleansing for Models Trained with SGD](https://mlsanity.com/paper/1906.08473/) (2019)
- [Influence Function based Data Poisoning Attacks to Top-N Recommender Systems](https://mlsanity.com/paper/2002.08025/) (2020)
- [Introduction to Core-sets: an Updated Survey](https://mlsanity.com/paper/2011.09384/) (2020)
- [Exploring Example Influence in Continual Learning](https://mlsanity.com/paper/2209.12241/) (2022)
- [PersonaX: A Recommendation Agent Oriented User Modeling Framework for Long Behavior Sequence](https://mlsanity.com/paper/2503.02398/) (2025)
- [Interpretable Data-Based Explanations for Fairness Debugging](https://mlsanity.com/paper/2112.09745/) (2021)
- [Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels](https://mlsanity.com/paper/2206.12569/) (2022)
- [Toward Optimal Probabilistic Active Learning Using a Bayesian Approach](https://mlsanity.com/paper/2006.01732/) (2020)
- [lpNTK: Better Generalisation with Less Data via Sample Interaction During Learning](https://mlsanity.com/paper/2401.08808/) (2024)
- [Data Efficiency for Large Recommendation Models](https://mlsanity.com/paper/2410.18111/) (2024)
- [Training-free Heterogeneous Graph Condensation via Data Selection](https://mlsanity.com/paper/2412.16250/) (2024)

## Built on by (most cited first, 1 of 1)

- [How Faithful Is Trajectory-Based Data Attribution? Error Sources, Remedies, and Practical Guidelines](https://mlsanity.com/paper/2605.18814/) (2026)

## Abstract (arXiv)

Modern data-centric AI needs precise per-sample influence. Standard SGD-IE approximates leave-one-out effects by summing per-epoch surrogates and ignores cross-epoch compounding, which misranks critical examples. We propose ACC-SGD-IE, a trajectory-aware estimator that propagates the leave-one-out perturbation across training and updates an accumulative influence state at each step. In smooth strongly convex settings it achieves geometric error contraction and, in smooth non-convex regimes, it tightens error bounds; larger mini-batches further reduce constants. Empirically, on Adult, 20 Newsgroups, and MNIST under clean and corrupted data and both convex and non-convex training, ACC-SGD-IE yields more accurate influence estimates, especially over long epochs. For downstream data cleansing it more reliably flags noisy samples, producing models trained on ACC-SGD-IE cleaned data that outperform those cleaned with SGD-IE.