# Data Cleansing for Models Trained with SGD

arXiv: https://arxiv.org/abs/1906.08473 · HTML: https://mlsanity.com/paper/1906.08473/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 30 of 67)

- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks](https://mlsanity.com/paper/2102.02515/) (2021)
- [Training Data Attribution via Approximate Unrolled Differentiation](https://mlsanity.com/paper/2405.12186/) (2024)
- [Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation](https://mlsanity.com/paper/2201.10055/) (2022)
- [Repairing Neural Networks by Leaving the Right Past Behind](https://mlsanity.com/paper/2207.04806/) (2022)
- [Understanding Instance-based Interpretability of Variational Auto-Encoders](https://mlsanity.com/paper/2105.14203/) (2021)
- [Towards Tracing Factual Knowledge in Language Models Back to the Training Data](https://mlsanity.com/paper/2205.11482/) (2022)
- [Token-wise Influential Training Data Retrieval for Large Language Models](https://mlsanity.com/paper/2405.11724/) (2024)
- [Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees](https://mlsanity.com/paper/2205.00359/) (2022)
- [The Memory Perturbation Equation: Understanding Model's Sensitivity to Data](https://mlsanity.com/paper/2310.19273/) (2023)
- [Natural Example-Based Explainability: a Survey](https://mlsanity.com/paper/2309.03234/) (2023)
- [Remaining-data-free Machine Unlearning by Suppressing Sample Contribution](https://mlsanity.com/paper/2402.15109/) (2024)
- [Contrastive Error Attribution for Finetuned Language Models](https://mlsanity.com/paper/2212.10722/) (2022)
- [Model Generalization on COVID-19 Fake News Detection](https://mlsanity.com/paper/2101.03841/) (2021)
- [Influence Estimation for Generative Adversarial Networks](https://mlsanity.com/paper/2101.08367/) (2021)
- [A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learning](https://mlsanity.com/paper/2505.19281/) (2025)
- [Sample based Explanations via Generalized Representers](https://mlsanity.com/paper/2310.18526/) (2023)
- [Better Training Data Attribution via Better Inverse Hessian-Vector Products](https://mlsanity.com/paper/2507.14740/) (2025)
- [Data Cleaning and Machine Learning: A Systematic Literature Review](https://mlsanity.com/paper/2310.01765/) (2023)
- [Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation](https://mlsanity.com/paper/2305.09651/) (2023)
- [GraSS: Scalable Data Attribution with Gradient Sparsification and Sparse Projection](https://mlsanity.com/paper/2505.18976/) (2025)
- [Neural Network Libraries: A Deep Learning Framework Designed from Engineers' Perspectives](https://mlsanity.com/paper/2102.06725/) (2021)
- [Finding High-Value Training Data Subset through Differentiable Convex Programming](https://mlsanity.com/paper/2104.13794/) (2021)
- [Influence Scores at Scale for Efficient Language Data Sampling](https://mlsanity.com/paper/2311.16298/) (2023)
- [Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods](https://mlsanity.com/paper/2412.03906/) (2024)
- [The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research](https://mlsanity.com/paper/2310.07882/) (2023)
- [How Faithful Is Trajectory-Based Data Attribution? Error Sources, Remedies, and Practical Guidelines](https://mlsanity.com/paper/2605.18814/) (2026)
- [Approximating Instance-Dependent Noise via Instance-Confidence Embedding](https://mlsanity.com/paper/2103.13569/) (2021)
- [UTrace: Poisoning Forensics for Private Collaborative Learning](https://mlsanity.com/paper/2409.15126/) (2024)

## Abstract (arXiv)

Data cleansing is a typical approach used to improve the accuracy of machine learning models, which, however, requires extensive domain knowledge to identify the influential instances that affect the models. In this paper, we propose an algorithm that can suggest influential instances without using any domain knowledge. With the proposed method, users only need to inspect the instances suggested by the algorithm, implying that users do not need extensive knowledge for this procedure, which enables even non-experts to conduct data cleansing and improve the model. The existing methods require the loss function to be convex and an optimal model to be obtained, which is not always the case in modern machine learning. To overcome these limitations, we propose a novel approach specifically designed for the models trained with stochastic gradient descent (SGD). The proposed method infers the influential instances by retracing the steps of the SGD while incorporating intermediate models computed in each step. Through experiments, we demonstrate that the proposed method can accurately infer the influential instances. Moreover, we used MNIST and CIFAR10 to show that the models can be effectively improved by removing the influential instances suggested by the proposed method.