# Stream-based Active Learning with Verification Latency in Non-stationary Environments

arXiv: https://arxiv.org/abs/2204.06822 · HTML: https://mlsanity.com/paper/2204.06822/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 24 references that are in the map.
- First posted on arXiv 14 Apr 2022.

## Builds on (most cited first, 2 of 2)

- [Comparative Analysis of Extreme Verification Latency Learning Algorithms](https://mlsanity.com/paper/2011.14917/) (2020)
- [Task-Sensitive Concept Drift Detector with Constraint Embedding](https://mlsanity.com/paper/2108.06980/) (2021)

## Built on by (most cited first, 5 of 5)

- [Active learning for data streams: a survey](https://mlsanity.com/paper/2302.08893/) (2023)
- [Streaming Active Learning for Regression Problems Using Regression via Classification](https://mlsanity.com/paper/2309.01013/) (2023)
- [Online Active Learning For Sound Event Detection](https://mlsanity.com/paper/2309.14460/) (2023)
- [AiGAS-dEVL: An Adaptive Incremental Neural Gas Model for Drifting Data Streams under Extreme Verification Latency](https://mlsanity.com/paper/2407.05379/) (2024)
- [Stream-Based Active Learning for Process Monitoring](https://mlsanity.com/paper/2411.12563/) (2024)

## Abstract (arXiv)

Data stream classification is an important problem in the field of machine learning. Due to the non-stationary nature of the data where the underlying distribution changes over time (concept drift), the model needs to continuously adapt to new data statistics. Stream-based Active Learning (AL) approaches address this problem by interactively querying a human expert to provide new data labels for the most recent samples, within a limited budget. Existing AL strategies assume that labels are immediately available, while in a real-world scenario the expert requires time to provide a queried label (verification latency), and by the time the requested labels arrive they may not be relevant anymore. In this article, we investigate the influence of finite, time-variable, and unknown verification delay, in the presence of concept drift on AL approaches. We propose PRopagate (PR), a latency independent utility estimator which also predicts the requested, but not yet known, labels. Furthermore, we propose a drift-dependent dynamic budget strategy, which uses a variable distribution of the labelling budget over time, after a detected drift. Thorough experimental evaluation, with both synthetic and real-world non-stationary datasets, and different settings of verification latency and budget are conducted and analyzed. We empirically show that the proposed method consistently outperforms the state-of-the-art. Additionally, we demonstrate that with variable budget allocation in time, it is possible to boost the performance of AL strategies, without increasing the overall labeling budget.