# Stopping criterion for active learning based on deterministic generalization bounds

arXiv: https://arxiv.org/abs/2005.07402 · HTML: https://mlsanity.com/paper/2005.07402/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2020.

## Built on by (most cited first, 20 of 20)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Hitting the Target: Stopping Active Learning at the Cost-Based Optimum](https://mlsanity.com/paper/2110.03802/) (2021)
- [Active Learning by Query by Committee with Robust Divergences](https://mlsanity.com/paper/2211.10013/) (2022)
- [On the Limitations of Simulating Active Learning](https://mlsanity.com/paper/2305.13342/) (2023)
- [Stopping Criterion for Active Learning Based on Error Stability](https://mlsanity.com/paper/2104.01836/) (2021)
- [TrustAL: Trustworthy Active Learning using Knowledge Distillation](https://mlsanity.com/paper/2201.11661/) (2022)
- [Robust online active learning](https://mlsanity.com/paper/2302.00422/) (2023)
- [Online Decision Mediation](https://mlsanity.com/paper/2310.18601/) (2023)
- [Cost-effective Framework for Gradual Domain Adaptation with Multifidelity](https://mlsanity.com/paper/2202.04359/) (2022)
- [Improving Probabilistic Models in Text Classification via Active Learning](https://mlsanity.com/paper/2202.02629/) (2022)
- [Bayesian inference of composition-dependent phase diagrams](https://mlsanity.com/paper/2309.01271/) (2023)
- [Knowledge Distillation under Teacher Misspecification: An Order-Parameter Analysis of the Gap between Teacher Mimicry and Task Performance](https://mlsanity.com/paper/2608.29472/) (2026)
- [Revisiting Sample Size Determination in Natural Language Understanding](https://mlsanity.com/paper/2307.00374/) (2023)
- [Batch Active Learning in Gaussian Process Regression using Derivatives](https://mlsanity.com/paper/2408.01861/) (2024)
- [Reciprocal Learning](https://mlsanity.com/paper/2408.06257/) (2024)
- [An (ε,δ)-accurate level set estimation with a stopping criterion](https://mlsanity.com/paper/2503.20272/) (2025)
- [Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement](https://mlsanity.com/paper/2504.16136/) (2025)
- [An Analysis of Active Learning Algorithms using Real-World Crowd-sourced Text Annotations](https://mlsanity.com/paper/2604.23290/) (2026)
- [Learnable Weighting of Intra-Attribute Distances for Categorical Data Clustering with Nominal and Ordinal Attributes](https://mlsanity.com/paper/2607.05464/) (2026)
- [A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networks](https://mlsanity.com/paper/2609.21017/) (2026)

## Abstract (arXiv)

Active learning is a framework in which the learning machine can select the samples to be used for training. This technique is promising, particularly when the cost of data acquisition and labeling is high. In active learning, determining the timing at which learning should be stopped is a critical issue. In this study, we propose a criterion for automatically stopping active learning. The proposed stopping criterion is based on the difference in the expected generalization errors and hypothesis testing. We derive a novel upper bound for the difference in expected generalization errors before and after obtaining a new training datum based on PAC-Bayesian theory. Unlike ordinary PAC-Bayesian bounds, though, the proposed bound is deterministic; hence, there is no uncontrollable trade-off between the confidence and tightness of the inequality. We combine the upper bound with a statistical test to derive a stopping criterion for active learning. We demonstrate the effectiveness of the proposed method via experiments with both artificial and real datasets.