# Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applications

arXiv: https://arxiv.org/abs/2110.13290 · HTML: https://mlsanity.com/paper/2110.13290/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 60 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Expert Gate: Lifelong Learning with a Network of Experts](https://mlsanity.com/paper/1611.06194/) (2016)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Overcoming Catastrophic Forgetting by Incremental Moment Matching](https://mlsanity.com/paper/1703.08475/) (2017)
- [Training Deep Neural Networks with 8-bit Floating Point Numbers](https://mlsanity.com/paper/1812.08011/) (2018)
- [Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables](https://mlsanity.com/paper/1604.08880/) (2016)
- [Deep Learning for Audio Signal Processing](https://mlsanity.com/paper/1905.00078/) (2019)
- [DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing](https://mlsanity.com/paper/1611.01942/) (2016)
- [Ensembles of Deep LSTM Learners for Activity Recognition using Wearables](https://mlsanity.com/paper/1703.09370/) (2017)
- [Mixed Precision Training of Convolutional Neural Networks using Integer Operations](https://mlsanity.com/paper/1802.00930/) (2018)
- [A comprehensive, application-oriented study of catastrophic forgetting in DNNs](https://mlsanity.com/paper/1905.08101/) (2019)
- [Continual Learning in Sensor-based Human Activity Recognition: an Empirical Benchmark Analysis](https://mlsanity.com/paper/2104.09396/) (2021)
- [FastICARL: Fast Incremental Classifier and Representation Learning with Efficient Budget Allocation in Audio Sensing Applications](https://mlsanity.com/paper/2106.07268/) (2021)
- [DSP.Ear: Leveraging Co-Processor Support for Continuous Audio Sensing on Smartphones](https://mlsanity.com/paper/1409.3206/) (2014)
- [Knowing when we do not know: Bayesian continual learning for sensing-based analysis tasks](https://mlsanity.com/paper/2106.05872/) (2021)

## Built on by (most cited first, 17 of 17)

- [LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platforms](https://mlsanity.com/paper/2311.11420/) (2023)
- [Cost-effective On-device Continual Learning over Memory Hierarchy with Miro](https://mlsanity.com/paper/2308.06053/) (2023)
- [Design Principles for Lifelong Learning AI Accelerators](https://mlsanity.com/paper/2310.04467/) (2023)
- [Class-incremental Learning for Time Series: Benchmark and Evaluation](https://mlsanity.com/paper/2402.12035/) (2024)
- [DOCTOR: A Multi-Disease Detection Continual Learning Framework Based on Wearable Medical Sensors](https://mlsanity.com/paper/2305.05738/) (2023)
- [Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learning](https://mlsanity.com/paper/2410.18378/) (2024)
- [Enabling On-Device Smartphone GPU based Training: Lessons Learned](https://mlsanity.com/paper/2202.10100/) (2022)
- [YONO: Modeling Multiple Heterogeneous Neural Networks on Microcontrollers](https://mlsanity.com/paper/2203.03794/) (2022)
- [REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning](https://mlsanity.com/paper/2406.04772/) (2024)
- [Enhancing Quantum Security over Federated Learning via Post-Quantum Cryptography](https://mlsanity.com/paper/2409.04637/) (2024)
- [PAGE: Domain-Incremental Adaptation with Past-Agnostic Generative Replay for Smart Healthcare](https://mlsanity.com/paper/2403.08197/) (2024)
- [ODEStream: A Buffer-Free Online Learning Framework with ODE-based Adaptor for Streaming Time Series Forecasting](https://mlsanity.com/paper/2411.07413/) (2024)
- [MetaCLBench: Meta Continual Learning Benchmark on Resource-Constrained Edge Devices](https://mlsanity.com/paper/2504.00174/) (2025)
- [Legilimens: Performant Video Analytics on the System-on-Chip Edge](https://mlsanity.com/paper/2504.21136/) (2025)
- [Responsive DNN Adaptation for Video Analytics against Environment Shift via Hierarchical Mobile-Cloud Collaborations](https://mlsanity.com/paper/2505.00745/) (2025)
- [Continual Learning at the Edge: An Agnostic IIoT Architecture](https://mlsanity.com/paper/2512.14311/) (2025)
- [What changes after deployment? A survey on On-device Learning in TinyML](https://mlsanity.com/paper/2605.31226/) (2026)

## Abstract (arXiv)

Continual learning approaches help deep neural network models adapt and learn incrementally by trying to solve catastrophic forgetting. However, whether these existing approaches, applied traditionally to image-based tasks, work with the same efficacy to the sequential time series data generated by mobile or embedded sensing systems remains an unanswered question. To address this void, we conduct the first comprehensive empirical study that quantifies the performance of three predominant continual learning schemes (i.e., regularization, replay, and replay with examples) on six datasets from three mobile and embedded sensing applications in a range of scenarios having different learning complexities. More specifically, we implement an end-to-end continual learning framework on edge devices. Then we investigate the generalizability, trade-offs between performance, storage, computational costs, and memory footprint of different continual learning methods. Our findings suggest that replay with exemplars-based schemes such as iCaRL has the best performance trade-offs, even in complex scenarios, at the expense of some storage space (few MBs) for training examples (1% to 5%). We also demonstrate for the first time that it is feasible and practical to run continual learning on-device with a limited memory budget. In particular, the latency on two types of mobile and embedded devices suggests that both incremental learning time (few seconds - 4 minutes) and training time (1 - 75 minutes) across datasets are acceptable, as training could happen on the device when the embedded device is charging thereby ensuring complete data privacy. Finally, we present some guidelines for practitioners who want to apply a continual learning paradigm for mobile sensing tasks.