# Efficient Estimation of Influence of a Training Instance

arXiv: https://arxiv.org/abs/2012.04207 · HTML: https://mlsanity.com/paper/2012.04207/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 11 of 11)

- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [Model Generalization on COVID-19 Fake News Detection](https://mlsanity.com/paper/2101.03841/) (2021)
- [Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation](https://mlsanity.com/paper/2112.08798/) (2021)
- [Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximation](https://mlsanity.com/paper/2108.10808/) (2021)
- [IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languages](https://mlsanity.com/paper/2311.12405/) (2023)
- [InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks](https://mlsanity.com/paper/2402.14702/) (2024)
- [Which Experiences Are Influential for RL Agents? Efficiently Estimating The Influence of Experiences](https://mlsanity.com/paper/2405.14629/) (2024)
- [LossVal: Efficient Data Valuation for Neural Networks](https://mlsanity.com/paper/2412.04158/) (2024)
- [Which Experiences Are Influential for Your Agent? Policy Iteration with Turn-over Dropout](https://mlsanity.com/paper/2301.11168/) (2023)

## Abstract (arXiv)

Understanding the influence of a training instance on a neural network model leads to improving interpretability. However, it is difficult and inefficient to evaluate the influence, which shows how a model's prediction would be changed if a training instance were not used. In this paper, we propose an efficient method for estimating the influence. Our method is inspired by dropout, which zero-masks a sub-network and prevents the sub-network from learning each training instance. By switching between dropout masks, we can use sub-networks that learned or did not learn each training instance and estimate its influence. Through experiments with BERT and VGGNet on classification datasets, we demonstrate that the proposed method can capture training influences, enhance the interpretability of error predictions, and cleanse the training dataset for improving generalization.