# Active Mini-Batch Sampling using Repulsive Point Processes

arXiv: https://arxiv.org/abs/1804.02772 · HTML: https://mlsanity.com/paper/1804.02772/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2018.

## Built on by (most cited first, 20 of 20)

- [The Effect of Diversity in Meta-Learning](https://mlsanity.com/paper/2201.11775/) (2022)
- [BigSmall: Efficient Multi-Task Learning for Disparate Spatial and Temporal Physiological Measurements](https://mlsanity.com/paper/2303.11573/) (2023)
- [Determinantal point processes based on orthogonal polynomials for sampling minibatches in SGD](https://mlsanity.com/paper/2112.06007/) (2021)
- [Batch Active Learning of Reward Functions from Human Preferences](https://mlsanity.com/paper/2402.15757/) (2024)
- [Uniform Sampling over Episode Difficulty](https://mlsanity.com/paper/2108.01662/) (2021)
- [A Novel Data Augmentation Technique for Out-of-Distribution Sample Detection using Compounded Corruptions](https://mlsanity.com/paper/2207.13916/) (2022)
- [How Important is Importance Sampling for Deep Budgeted Training?](https://mlsanity.com/paper/2110.14283/) (2021)
- [NAAQA: A Neural Architecture for Acoustic Question Answering](https://mlsanity.com/paper/2106.06147/) (2021)
- [DPP-based Client Selection for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2303.17358/) (2023)
- [Variance Matters: Improving Domain Adaptation via Stratified Sampling](https://mlsanity.com/paper/2512.05226/) (2025)
- [Leverage Score Sampling for Complete Mode Coverage in Generative Adversarial Networks](https://mlsanity.com/paper/2104.02373/) (2021)
- [Order Matters: Improving Domain Adaptation by Reordering Data](https://mlsanity.com/paper/2605.05084/) (2026)
- [Policy Gradient with Kernel Quadrature](https://mlsanity.com/paper/2310.14768/) (2023)
- [Online Importance Sampling for Stochastic Gradient Optimization](https://mlsanity.com/paper/2311.14468/) (2023)
- [Multiple Importance Sampling for Stochastic Gradient Estimation](https://mlsanity.com/paper/2407.15525/) (2024)
- [Transfer Learning with Active Sampling for Rapid Training and Calibration in BCI-P300 Across Health States and Multi-centre Data](https://mlsanity.com/paper/2412.17833/) (2024)
- [DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation](https://mlsanity.com/paper/2509.16173/) (2025)
- [Variance-reduced Domain Adaptation using Paired Sampling](https://mlsanity.com/paper/2607.20367/) (2026)
- [Online Variance Reduction for Domain Adaptation on Streaming Data](https://mlsanity.com/paper/2607.20374/) (2026)
- [Scalable Learning of Determinantal Point Processes with Kernel-Structured Interactions](https://mlsanity.com/paper/2610.12829/) (2026)

## Abstract (arXiv)

The convergence speed of stochastic gradient descent (SGD) can be improved by actively selecting mini-batches. We explore sampling schemes where similar data points are less likely to be selected in the same mini-batch. In particular, we prove that such repulsive sampling schemes lowers the variance of the gradient estimator. This generalizes recent work on using Determinantal Point Processes (DPPs) for mini-batch diversification (Zhang et al., 2017) to the broader class of repulsive point processes. We first show that the phenomenon of variance reduction by diversified sampling generalizes in particular to non-stationary point processes. We then show that other point processes may be computationally much more efficient than DPPs. In particular, we propose and investigate Poisson Disk sampling---frequently encountered in the computer graphics community---for this task. We show empirically that our approach improves over standard SGD both in terms of convergence speed as well as final model performance.