# Optimal Sub-sampling with Influence Functions

arXiv: https://arxiv.org/abs/1709.01716 · HTML: https://mlsanity.com/paper/1709.01716/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2017.

## Built on by (most cited first, 11 of 11)

- [Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning](https://mlsanity.com/paper/2110.14049/) (2021)
- [Towards a statistical theory of data selection under weak supervision](https://mlsanity.com/paper/2309.14563/) (2023)
- [Understanding Instance-based Interpretability of Variational Auto-Encoders](https://mlsanity.com/paper/2105.14203/) (2021)
- [Quality meets Diversity: A Model-Agnostic Framework for Computerized Adaptive Testing](https://mlsanity.com/paper/2101.05986/) (2021)
- [How Many and Which Training Points Would Need to be Removed to Flip this Prediction?](https://mlsanity.com/paper/2302.02169/) (2023)
- [DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Tasks](https://mlsanity.com/paper/2502.00270/) (2025)
- [A Coreset Learning Reality Check](https://mlsanity.com/paper/2301.06163/) (2023)
- [On Influence Functions, Classification Influence, Relative Influence, Memorization and Generalization](https://mlsanity.com/paper/2305.16094/) (2023)
- [Graph-Based Model-Agnostic Data Subsampling for Recommendation Systems](https://mlsanity.com/paper/2305.16391/) (2023)
- [InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks](https://mlsanity.com/paper/2402.14702/) (2024)
- [Backbone Augmented Training for Adaptations](https://mlsanity.com/paper/2506.04288/) (2025)

## Abstract (arXiv)

Sub-sampling is a common and often effective method to deal with the computational challenges of large datasets. However, for most statistical models, there is no well-motivated approach for drawing a non-uniform subsample. We show that the concept of an asymptotically linear estimator and the associated influence function leads to optimal sampling procedures for a wide class of popular models. Furthermore, for linear regression models which have well-studied procedures for non-uniform sub-sampling, we show our optimal influence function based method outperforms previous approaches. We empirically show the improved performance of our method on real datasets.