# ORIS: Online Active Learning Using Reinforcement Learning-based Inclusive Sampling for Robust Streaming Analytics System

arXiv: https://arxiv.org/abs/2411.18060 · HTML: https://mlsanity.com/paper/2411.18060/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Cost-Effective Active Learning for Deep Image Classification](https://mlsanity.com/paper/1701.03551/) (2017)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [Learning how to Active Learn: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1708.02383/) (2017)
- [Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion](https://mlsanity.com/paper/1804.07745/) (2018)
- [Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics](https://mlsanity.com/paper/2110.01518/) (2021)
- [Reinforced active learning for image segmentation](https://mlsanity.com/paper/2002.06583/) (2020)
- [Modeling and mitigating human annotation errors to design efficient stream processing systems with human-in-the-loop machine learning](https://mlsanity.com/paper/2007.03177/) (2020)
- [Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1906.03098/) (2019)
- [Modeling Human Annotation Errors to Design Bias-Aware Systems for Social Stream Processing](https://mlsanity.com/paper/1907.07228/) (2019)
- [Engineering Crowdsourced Stream Processing Systems](https://mlsanity.com/paper/1310.5463/) (2013)

## Abstract (arXiv)

Effective labeled data collection plays a critical role in developing and fine-tuning robust streaming analytics systems. However, continuously labeling documents to filter relevant information poses significant challenges like limited labeling budget or lack of high-quality labels. There is a need for efficient human-in-the-loop machine learning (HITL-ML) design to improve streaming analytics systems. One particular HITL- ML approach is online active learning, which involves iteratively selecting a small set of the most informative documents for labeling to enhance the ML model performance. The performance of such algorithms can get affected due to human errors in labeling. To address these challenges, we propose ORIS, a method to perform Online active learning using Reinforcement learning-based Inclusive Sampling of documents for labeling. ORIS aims to create a novel Deep Q-Network-based strategy to sample incoming documents that minimize human errors in labeling and enhance the ML model performance. We evaluate the ORIS method on emotion recognition tasks, and it outperforms traditional baselines in terms of both human labeling performance and the ML model performance.