# Multi-Label Adaptive Batch Selection by Highlighting Hard and Imbalanced Samples

arXiv: https://arxiv.org/abs/2403.18192 · HTML: https://mlsanity.com/paper/2403.18192/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples](https://mlsanity.com/paper/1704.07433/) (2017)
- [Online Batch Selection for Faster Training of Neural Networks](https://mlsanity.com/paper/1511.06343/) (2015)
- [Should we really use post-hoc tests based on mean-ranks?](https://mlsanity.com/paper/1505.02288/) (2015)
- [Hard Sample Aware Network for Contrastive Deep Graph Clustering](https://mlsanity.com/paper/2212.08665/) (2022)
- [Disentangled Variational Autoencoder based Multi-Label Classification with Covariance-Aware Multivariate Probit Model](https://mlsanity.com/paper/2007.06126/) (2020)
- [Carpe Diem, Seize the Samples Uncertain "At the Moment" for Adaptive Batch Selection](https://mlsanity.com/paper/1911.08050/) (2019)
- [Multi-Label Sampling based on Local Label Imbalance](https://mlsanity.com/paper/2005.03240/) (2020)
- [Making Classifier Chains Resilient to Class Imbalance](https://mlsanity.com/paper/1807.11393/) (2018)

## Built on by (most cited first, 5 of 5)

- [Active Data Curation Effectively Distills Large-Scale Multimodal Models](https://mlsanity.com/paper/2411.18674/) (2024)
- [Batch Selection for Multi-Label Classification Guided by Uncertainty and Dynamic Label Correlations](https://mlsanity.com/paper/2412.16521/) (2024)
- [D2ACE: Multi-Label Batch Selection Guided by Dual Dynamics and Adaptive Correlation Enhancement](https://mlsanity.com/paper/2605.09400/) (2026)
- [Addressing Imbalance in Multi-Label Data via Label-Specific Distance-based Oversampling](https://mlsanity.com/paper/2606.05927/) (2026)
- [Self-Explainable Multi-Label Graph Neural Network for Correlated Evidence Attribution](https://mlsanity.com/paper/2608.27574/) (2026)

## Abstract (arXiv)

Deep neural network models have demonstrated their effectiveness in classifying multi-label data from various domains. Typically, they employ a training mode that combines mini-batches with optimizers, where each sample is randomly selected with equal probability when constructing mini-batches. However, the intrinsic class imbalance in multi-label data may bias the model towards majority labels, since samples relevant to minority labels may be underrepresented in each mini-batch. Meanwhile, during the training process, we observe that instances associated with minority labels tend to induce greater losses. Existing heuristic batch selection methods, such as priority selection of samples with high contribution to the objective function, i.e., samples with high loss, have been proven to accelerate convergence while reducing the loss and test error in single-label data. However, batch selection methods have not yet been applied and validated in multi-label data. In this study, we introduce a simple yet effective adaptive batch selection algorithm tailored to multi-label deep learning models. It adaptively selects each batch by prioritizing hard samples related to minority labels. A variant of our method also takes informative label correlations into consideration. Comprehensive experiments combining five multi-label deep learning models on thirteen benchmark datasets show that our method converges faster and performs better than random batch selection.