# Less Is Better: Unweighted Data Subsampling via Influence Function

arXiv: https://arxiv.org/abs/1912.01321 · HTML: https://mlsanity.com/paper/1912.01321/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2019.

## Built on by (most cited first, 30 of 30)

- [MedCLIP: Contrastive Learning from Unpaired Medical Images and Text](https://mlsanity.com/paper/2210.10163/) (2022)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [PAC-Bayes Information Bottleneck](https://mlsanity.com/paper/2109.14509/) (2021)
- [Towards a statistical theory of data selection under weak supervision](https://mlsanity.com/paper/2309.14563/) (2023)
- [MediTab: Scaling Medical Tabular Data Predictors via Data Consolidation, Enrichment, and Refinement](https://mlsanity.com/paper/2305.12081/) (2023)
- [Regularizing Second-Order Influences for Continual Learning](https://mlsanity.com/paper/2304.10177/) (2023)
- [Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection](https://mlsanity.com/paper/2111.14932/) (2021)
- [Characterizing the Influence of Graph Elements](https://mlsanity.com/paper/2210.07441/) (2022)
- [From Points to Coalitions: Hierarchical Contrastive Shapley Values for Prioritizing Data Samples](https://mlsanity.com/paper/2512.19363/) (2025)
- [Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective](https://mlsanity.com/paper/2401.04374/) (2024)
- [Understanding Programmatic Weak Supervision via Source-aware Influence Function](https://mlsanity.com/paper/2205.12879/) (2022)
- [Deeper Understanding of Black-box Predictions via Generalized Influence Functions](https://mlsanity.com/paper/2312.05586/) (2023)
- [Energy-based Unknown Intent Detection with Data Manipulation](https://mlsanity.com/paper/2107.12542/) (2021)
- [DIWIFT: Discovering Instance-wise Influential Features for Tabular Data](https://mlsanity.com/paper/2207.02773/) (2022)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [A Survey of Dataset Refinement for Problems in Computer Vision Datasets](https://mlsanity.com/paper/2210.11717/) (2022)
- [Improve Knowledge Distillation via Label Revision and Data Selection](https://mlsanity.com/paper/2404.03693/) (2024)
- [The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research](https://mlsanity.com/paper/2310.07882/) (2023)
- [A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective](https://mlsanity.com/paper/2411.00360/) (2024)
- [Train on Validation (ToV): Fast data selection with applications to fine-tuning](https://mlsanity.com/paper/2510.00386/) (2025)
- [Estimation of Local Average Treatment Effect by Data Combination](https://mlsanity.com/paper/2109.05175/) (2021)
- [PruNeRF: Segment-Centric Dataset Pruning via 3D Spatial Consistency](https://mlsanity.com/paper/2406.00798/) (2024)
- [Influence-based Attributions can be Manipulated](https://mlsanity.com/paper/2409.05208/) (2024)
- [On Influence Functions, Classification Influence, Relative Influence, Memorization and Generalization](https://mlsanity.com/paper/2305.16094/) (2023)
- [CLIMAX: An exploration of Classifier-Based Contrastive Explanations](https://mlsanity.com/paper/2307.00680/) (2023)
- [InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks](https://mlsanity.com/paper/2402.14702/) (2024)
- [Robust Noisy Label Learning via Two-Stream Sample Distillation](https://mlsanity.com/paper/2404.10499/) (2024)
- [DeGLIF for Label Noise Robust Node Classification using GNNs](https://mlsanity.com/paper/2506.00244/) (2025)
- [Knowledge Distillation with Adapted Weight](https://mlsanity.com/paper/2501.02705/) (2025)
- [Backbone Augmented Training for Adaptations](https://mlsanity.com/paper/2506.04288/) (2025)

## Abstract (arXiv)

In the time of Big Data, training complex models on large-scale data sets is challenging, making it appealing to reduce data volume for saving computation resources by subsampling. Most previous works in subsampling are weighted methods designed to help the performance of subset-model approach the full-set-model, hence the weighted methods have no chance to acquire a subset-model that is better than the full-set-model. However, we question that how can we achieve better model with less data? In this work, we propose a novel Unweighted Influence Data Subsampling (UIDS) method, and prove that the subset-model acquired through our method can outperform the full-set-model. Besides, we show that overly confident on a given test set for sampling is common in Influence-based subsampling methods, which can eventually cause our subset-model's failure in out-of-sample test. To mitigate it, we develop a probabilistic sampling scheme to control the worst-case risk over all distributions close to the empirical distribution. The experiment results demonstrate our methods superiority over existed subsampling methods in diverse tasks, such as text classification, image classification, click-through prediction, etc.