# Carpe Diem, Seize the Samples Uncertain "At the Moment" for Adaptive Batch Selection

arXiv: https://arxiv.org/abs/1911.08050 · HTML: https://mlsanity.com/paper/1911.08050/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 10 of 10)

- [Rho-1: Not All Tokens Are What You Need](https://mlsanity.com/paper/2404.07965/) (2024)
- [Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay](https://mlsanity.com/paper/2506.05316/) (2025)
- [Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty](https://mlsanity.com/paper/2202.01339/) (2022)
- [Active Data Curation Effectively Distills Large-Scale Multimodal Models](https://mlsanity.com/paper/2411.18674/) (2024)
- [Active Generation for Image Classification](https://mlsanity.com/paper/2403.06517/) (2024)
- [Multi-Label Adaptive Batch Selection by Highlighting Hard and Imbalanced Samples](https://mlsanity.com/paper/2403.18192/) (2024)
- [Batch Selection for Multi-Label Classification Guided by Uncertainty and Dynamic Label Correlations](https://mlsanity.com/paper/2412.16521/) (2024)
- [Bandit-Driven Batch Selection for Robust Learning under Label Noise](https://mlsanity.com/paper/2311.00096/) (2023)
- [Improve Deep Image Inpainting by Emphasizing the Complexity of Missing Regions](https://mlsanity.com/paper/2202.06266/) (2022)
- [D2ACE: Multi-Label Batch Selection Guided by Dual Dynamics and Adaptive Correlation Enhancement](https://mlsanity.com/paper/2605.09400/) (2026)

## Abstract (arXiv)

The accuracy of deep neural networks is significantly affected by how well mini-batches are constructed during the training step. In this paper, we propose a novel adaptive batch selection algorithm called Recency Bias that exploits the uncertain samples predicted inconsistently in recent iterations. The historical label predictions of each training sample are used to evaluate its predictive uncertainty within a sliding window. Then, the sampling probability for the next mini-batch is assigned to each training sample in proportion to its predictive uncertainty. By taking advantage of this design, Recency Bias not only accelerates the training step but also achieves a more accurate network. We demonstrate the superiority of Recency Bias by extensive evaluation on two independent tasks. Compared with existing batch selection methods, the results showed that Recency Bias reduced the test error by up to 20.97% in a fixed wall-clock training time. At the same time, it improved the training time by up to 59.32% to reach the same test error