# Active-Learning-as-a-Service: An Automatic and Efficient MLOps System for Data-Centric AI

arXiv: https://arxiv.org/abs/2207.09109 · HTML: https://mlsanity.com/paper/2207.09109/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 58 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [Contextual Diversity for Active Learning](https://mlsanity.com/paper/2008.05723/) (2020)
- [Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets](https://mlsanity.com/paper/2202.02794/) (2022)
- [Diverse mini-batch Active Learning](https://mlsanity.com/paper/1901.05954/) (2019)
- [Sequential Graph Convolutional Network for Active Learning](https://mlsanity.com/paper/2006.10219/) (2020)
- [modAL: A modular active learning framework for Python](https://mlsanity.com/paper/1805.00979/) (2018)
- [A Data Quality-Driven View of MLOps](https://mlsanity.com/paper/2102.07750/) (2021)
- [Ludwig: a type-based declarative deep learning toolbox](https://mlsanity.com/paper/1909.07930/) (2019)
- [BoostClean: Automated Error Detection and Repair for Machine Learning](https://mlsanity.com/paper/1711.01299/) (2017)
- [Amazon SageMaker Autopilot: a white box AutoML solution at scale](https://mlsanity.com/paper/2012.08483/) (2020)
- [MLModelCI: An Automatic Cloud Platform for Efficient MLaaS](https://mlsanity.com/paper/2006.05096/) (2020)
- [AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly](https://mlsanity.com/paper/2105.10762/) (2021)
- [DeepAL: Deep Active Learning in Python](https://mlsanity.com/paper/2111.15258/) (2021)
- [libact: Pool-based Active Learning in Python](https://mlsanity.com/paper/1710.00379/) (2017)
- [ALiPy: Active Learning in Python](https://mlsanity.com/paper/1901.03802/) (2019)
- [Continuous Integration of Machine Learning Models with ease.ml/ci: Towards a Rigorous Yet Practical Treatment](https://mlsanity.com/paper/1903.00278/) (2019)
- [ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems](https://mlsanity.com/paper/2106.03122/) (2021)
- [DataCLUE: A Benchmark Suite for Data-centric NLP](https://mlsanity.com/paper/2111.08647/) (2021)
- [Hysia: Serving DNN-Based Video-to-Retail Applications in Cloud](https://mlsanity.com/paper/2006.05117/) (2020)
- [YMIR: A Rapid Data-centric Development Platform for Vision Applications](https://mlsanity.com/paper/2111.10046/) (2021)
- [SMART: An Open Source Data Labeling Platform for Supervised Learning](https://mlsanity.com/paper/1812.06591/) (2018)
- [Automatic Knowledge Augmentation for Generative Commonsense Reasoning](https://mlsanity.com/paper/2111.00192/) (2021)

## Abstract (arXiv)

The success of today's AI applications requires not only model training (Model-centric) but also data engineering (Data-centric). In data-centric AI, active learning (AL) plays a vital role, but current AL tools 1) require users to manually select AL strategies, and 2) can not perform AL tasks efficiently. To this end, this paper presents an automatic and efficient MLOps system for AL, named ALaaS (Active-Learning-as-a-Service). Specifically, 1) ALaaS implements an AL agent, including a performance predictor and a workflow controller, to decide the most suitable AL strategies given users' datasets and budgets. We call this a predictive-based successive halving early-stop (PSHEA) procedure. 2) ALaaS adopts a server-client architecture to support an AL pipeline and implements stage-level parallelism for high efficiency. Meanwhile, caching and batching techniques are employed to further accelerate the AL process. In addition to efficiency, ALaaS ensures accessibility with the help of the design philosophy of configuration-as-a-service. Extensive experiments show that ALaaS outperforms all other baselines in terms of latency and throughput. Also, guided by the AL agent, ALaaS can automatically select and run AL strategies for non-expert users under different datasets and budgets. Our code is available at urlhttps://github.com/MLSysOps/Active-Learning-as-a-Service.