# Submodular Batch Selection for Training Deep Neural Networks

arXiv: https://arxiv.org/abs/1906.08771 · HTML: https://mlsanity.com/paper/1906.08771/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 7 of 7)

- [S3VAADA: Submodular Subset Selection for Virtual Adversarial Active Domain Adaptation](https://mlsanity.com/paper/2109.08901/) (2021)
- [Active Data Curation Effectively Distills Large-Scale Multimodal Models](https://mlsanity.com/paper/2411.18674/) (2024)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [Less is more: Selecting informative and diverse subsets with balancing constraints](https://mlsanity.com/paper/2104.12835/) (2021)
- [A Fast and Effective Method for Euclidean Anticlustering: The Assignment-Based-Anticlustering Algorithm](https://mlsanity.com/paper/2601.06351/) (2026)
- [Scalable Distributed Algorithms for Size-Constrained Submodular Maximization in the MapReduce and Adaptive Complexity Models](https://mlsanity.com/paper/2206.09563/) (2022)
- [Fusing Conditional Submodular GAN and Programmatic Weak Supervision](https://mlsanity.com/paper/2312.10366/) (2023)

## Abstract (arXiv)

Mini-batch gradient descent based methods are the de facto algorithms for training neural network architectures today. We introduce a mini-batch selection strategy based on submodular function maximization. Our novel submodular formulation captures the informativeness of each sample and diversity of the whole subset. We design an efficient, greedy algorithm which can give high-quality solutions to this NP-hard combinatorial optimization problem. Our extensive experiments on standard datasets show that the deep models trained using the proposed batch selection strategy provide better generalization than Stochastic Gradient Descent as well as a popular baseline sampling strategy across different learning rates, batch sizes, and distance metrics.