# ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems

arXiv: https://arxiv.org/abs/2106.03122 · HTML: https://mlsanity.com/paper/2106.03122/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Don't forget, there is more than forgetting: new metrics for Continual Learning](https://mlsanity.com/paper/1810.13166/) (2018)
- [River: machine learning for streaming data in Python](https://mlsanity.com/paper/2012.04740/) (2020)
- [TensorFlow-Serving: Flexible, High-Performance ML Serving](https://mlsanity.com/paper/1712.06139/) (2017)
- [Monitoring and explainability of models in production](https://mlsanity.com/paper/2007.06299/) (2020)
- [MLModelCI: An Automatic Cloud Platform for Efficient MLaaS](https://mlsanity.com/paper/2006.05096/) (2020)
- [A Serverless Cloud-Fog Platform for DNN-Based Video Analytics with Incremental Learning](https://mlsanity.com/paper/2102.03012/) (2021)
- [ResumeNet: A Learning-based Framework for Automatic Resume Quality Assessment](https://mlsanity.com/paper/1810.02832/) (2018)

## Built on by (most cited first, 9 of 9)

- [Management of Machine Learning Lifecycle Artifacts: A Survey](https://mlsanity.com/paper/2210.11831/) (2022)
- [Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges](https://mlsanity.com/paper/2207.11759/) (2022)
- [Progressive Continual Learning for Spoken Keyword Spotting](https://mlsanity.com/paper/2201.12546/) (2022)
- [Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting](https://mlsanity.com/paper/2203.16361/) (2022)
- [Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive Models](https://mlsanity.com/paper/2206.06957/) (2022)
- [Continual Learning For On-Device Environmental Sound Classification](https://mlsanity.com/paper/2207.07429/) (2022)
- [Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach](https://mlsanity.com/paper/2410.21346/) (2024)
- [Active-Learning-as-a-Service: An Automatic and Efficient MLOps System for Data-Centric AI](https://mlsanity.com/paper/2207.09109/) (2022)
- [DataCI: A Platform for Data-Centric AI on Streaming Data](https://mlsanity.com/paper/2306.15538/) (2023)

## Abstract (arXiv)

MLOps is about taking experimental ML models to production, i.e., serving the models to actual users. Unfortunately, existing ML serving systems do not adequately handle the dynamic environments in which online data diverges from offline training data, resulting in tedious model updating and deployment works. This paper implements a lightweight MLOps plugin, termed ModelCI-e (continuous integration and evolution), to address the issue. Specifically, it embraces continual learning (CL) and ML deployment techniques, providing end-to-end supports for model updating and validation without serving engine customization. ModelCI-e includes 1) a model factory that allows CL researchers to prototype and benchmark CL models with ease, 2) a CL backend to automate and orchestrate the model updating efficiently, and 3) a web interface for an ML team to manage CL service collaboratively. Our preliminary results demonstrate the usability of ModelCI-e, and indicate that eliminating the interference between model updating and inference workloads is crucial for higher system efficiency.