# On Influence Functions, Classification Influence, Relative Influence, Memorization and Generalization

arXiv: https://arxiv.org/abs/2305.16094 · HTML: https://mlsanity.com/paper/2305.16094/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 6 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 4 of 4)

- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Less Is Better: Unweighted Data Subsampling via Influence Function](https://mlsanity.com/paper/1912.01321/) (2019)
- [Finding Influential Instances for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2009.09841/) (2020)
- [Optimal Sub-sampling with Influence Functions](https://mlsanity.com/paper/1709.01716/) (2017)

## Built on by (most cited first, 1 of 1)

- [Fairshare Data Pricing via Data Valuation for Large Language Models](https://mlsanity.com/paper/2502.00198/) (2025)

## Abstract (arXiv)

Machine learning systems such as large scale recommendation systems or natural language processing systems are usually trained on billions of training points and are associated with hundreds of billions or trillions of parameters. Improving the learning process in such a way that both the training load is reduced and the model accuracy improved is highly desired. In this paper we take a first step toward solving this problem, studying influence functions from the perspective of simplifying the computations they involve. We discuss assumptions, under which influence computations can be performed on significantly fewer parameters. We also demonstrate that the sign of the influence value can indicate whether a training point is to memorize, as opposed to generalize upon. For this purpose we formally define what memorization means for a training point, as opposed to generalization. We conclude that influence functions can be made practical, even for large scale machine learning systems, and that influence values can be taken into account by algorithms that selectively remove training points, as part of the learning process.