# Continual Learning in Practice

arXiv: https://arxiv.org/abs/1903.05202 · HTML: https://mlsanity.com/paper/1903.05202/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2019.

## Built on by (most cited first, 28 of 28)

- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Continual Learning with Foundation Models: An Empirical Study of Latent Replay](https://mlsanity.com/paper/2205.00329/) (2022)
- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [Multiplayer Performative Prediction: Learning in Decision-Dependent Games](https://mlsanity.com/paper/2201.03398/) (2022)
- [DualNet: Continual Learning, Fast and Slow](https://mlsanity.com/paper/2110.00175/) (2021)
- [Continual Learning with Bayesian Model based on a Fixed Pre-trained Feature Extractor](https://mlsanity.com/paper/2204.13349/) (2022)
- [Knowledge-Adaptation Priors](https://mlsanity.com/paper/2106.08769/) (2021)
- [Continual Learning in Deep Networks: an Analysis of the Last Layer](https://mlsanity.com/paper/2106.01834/) (2021)
- [On the Diminishing Returns of Width for Continual Learning](https://mlsanity.com/paper/2403.06398/) (2024)
- [Continual Learning for Predictive Maintenance: Overview and Challenges](https://mlsanity.com/paper/2301.12467/) (2023)
- [Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap](https://mlsanity.com/paper/2201.11676/) (2022)
- [Lifelong Continual Learning for Anomaly Detection: New Challenges, Perspectives, and Insights](https://mlsanity.com/paper/2303.07557/) (2023)
- [Explanation Shift: Detecting distribution shifts on tabular data via the explanation space](https://mlsanity.com/paper/2210.12369/) (2022)
- [Explanation Shift: How Did the Distribution Shift Impact the Model?](https://mlsanity.com/paper/2303.08081/) (2023)
- [Desiderata for Explainable AI in statistical production systems of the European Central Bank](https://mlsanity.com/paper/2107.08045/) (2021)
- [An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context](https://mlsanity.com/paper/2204.12781/) (2022)
- [Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive Models](https://mlsanity.com/paper/2206.06957/) (2022)
- [Modyn: Data-Centric Machine Learning Pipeline Orchestration](https://mlsanity.com/paper/2312.06254/) (2023)
- [Sequence Transferability and Task Order Selection in Continual Learning](https://mlsanity.com/paper/2502.06544/) (2025)
- [Towards better data discovery and collection with flow-based programming](https://mlsanity.com/paper/2108.04105/) (2021)
- [On The Reliability Of Machine Learning Applications In Manufacturing Environments](https://mlsanity.com/paper/2112.06986/) (2021)
- [Hardware-agnostic Computation for Large-scale Knowledge Graph Embeddings](https://mlsanity.com/paper/2207.08544/) (2022)
- [Balancing Continual Learning and Fine-tuning for Human Activity Recognition](https://mlsanity.com/paper/2401.02255/) (2024)
- [Towards Stable Machine Learning Model Retraining via Slowly Varying Sequences](https://mlsanity.com/paper/2403.19871/) (2024)
- [Introducing explainable supervised machine learning into interactive feedback loops for statistical production system](https://mlsanity.com/paper/2202.03212/) (2022)
- [Improving Feature Generalizability with Multitask Learning in Class Incremental Learning](https://mlsanity.com/paper/2204.12915/) (2022)
- [Continual Learning, Fast and Slow](https://mlsanity.com/paper/2209.02370/) (2022)
- [Development of an End-to-end Machine Learning System with Application to In-app Purchases](https://mlsanity.com/paper/2412.12390/) (2024)

## Abstract (arXiv)

This paper describes a reference architecture for self-maintaining systems that can learn continually, as data arrives. In environments where data evolves, we need architectures that manage Machine Learning (ML) models in production, adapt to shifting data distributions, cope with outliers, retrain when necessary, and adapt to new tasks. This represents continual AutoML or Automatically Adaptive Machine Learning. We describe the challenges and proposes a reference architecture.