# Empowering Active Learning to Jointly Optimize System and User Demands

arXiv: https://arxiv.org/abs/2005.04470 · HTML: https://mlsanity.com/paper/2005.04470/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2020.

## Built on by (most cited first, 10 of 10)

- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [Adapting Coreference Resolution Models through Active Learning](https://mlsanity.com/paper/2104.07611/) (2021)
- [LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework](https://mlsanity.com/paper/2112.07522/) (2021)
- [Annotation Curricula to Implicitly Train Non-Expert Annotators](https://mlsanity.com/paper/2106.02382/) (2021)
- [Implementing Active Learning in Cybersecurity: Detecting Anomalies in Redacted Emails](https://mlsanity.com/paper/2303.00870/) (2023)
- [Controlling Cloze-test Question Item Difficulty with PLM-based Surrogate Models for IRT Assessment](https://mlsanity.com/paper/2403.01456/) (2024)
- [Maximizing Information Gain in Privacy-Aware Active Learning of Email Anomalies](https://mlsanity.com/paper/2405.07440/) (2024)
- [Constrained C-Test Generation via Mixed-Integer Programming](https://mlsanity.com/paper/2404.08821/) (2024)
- [Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement](https://mlsanity.com/paper/2504.16136/) (2025)
- [FunnelAL: Retrieve-then-Rank Active Learning for Single-Class Discovery](https://mlsanity.com/paper/2607.25276/) (2026)

## Abstract (arXiv)

Existing approaches to active learning maximize the system performance by sampling unlabeled instances for annotation that yield the most efficient training. However, when active learning is integrated with an end-user application, this can lead to frustration for participating users, as they spend time labeling instances that they would not otherwise be interested in reading. In this paper, we propose a new active learning approach that jointly optimizes the seemingly counteracting objectives of the active learning system (training efficiently) and the user (receiving useful instances). We study our approach in an educational application, which particularly benefits from this technique as the system needs to rapidly learn to predict the appropriateness of an exercise to a particular user, while the users should receive only exercises that match their skills. We evaluate multiple learning strategies and user types with data from real users and find that our joint approach better satisfies both objectives when alternative methods lead to many unsuitable exercises for end users.