mlsanityData to 7 Oct 2026

Online Continual Learning for Embedded Devices

arXiv 2203.10681 ↗First posted 21 Mar 2022cs.LG cs.AIcited by 48

Not on the map. Why?

Abstract (arXiv)

Real-time on-device continual learning is needed for new applications such as home robots, user personalization on smartphones, and augmented/virtual reality headsets. However, this setting poses unique challenges: embedded devices have limited memory and compute capacity and conventional machine learning models suffer from catastrophic forgetting when updated on non-stationary data streams. While several online continual learning models have been developed, their effectiveness for embedded applications has not been rigorously studied. In this paper, we first identify criteria that online continual learners must meet to effectively perform real-time, on-device learning. We then study the efficacy of several online continual learning methods when used with mobile neural networks. We measure their performance, memory usage, compute requirements, and ability to generalize to out-of-domain inputs.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 75 of its 96 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

75 of 96 references.

References · 75

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2016Learning without Forgettingcited by 2,465
38 more
  1. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  2. 2016Matching Networks for One Shot Learningcited by 2,246
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  4. 2019Searching for MobileNetV3cited by 1,984
  5. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  8. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  9. 2016Progressive Neural Networkscited by 1,332
  10. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  12. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  13. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  14. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  15. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  16. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  17. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  18. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  19. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  20. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  21. 2019Large Scale Incremental Learningcited by 727
  22. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  23. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  24. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  25. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  26. 2018End-to-End Incremental Learningcited by 596
  27. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  28. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  29. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  30. 2017Progressive Neural Architecture Searchcited by 483
  31. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 287
  32. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  33. 2018Learning without Memorizingcited by 249
  34. 2020Few-Shot Class-Incremental Learningcited by 249
  35. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  36. 2015Quantized Convolutional Neural Networks for Mobile Devicescited by 212
  37. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  38. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 197

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 18; 2024: 15; 2025: 7; 2026: 4.

48 papers.

Sort:

By area:

  1. 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 120
  2. 2023Continual Learning: Applications and the Road Forwardcited by 59
  3. 2024Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learningcited by 49
  4. 2023Real-Time Evaluation in Online Continual Learning: A New Hopecited by 47
  5. 2024Gradient Reweighting: Towards Imbalanced Class-Incremental Learningcited by 37
  6. 2024Weight Clipping for Deep Continual and Reinforcement Learningcited by 29
  7. 2023SIESTA: Efficient Online Continual Learning with Sleepcited by 24
  8. 2023How Efficient Are Today's Continual Learning Algorithms?cited by 24
  9. 2024Streaming Deep Reinforcement Learning Finally Workscited by 24
  10. 2024Online Continual Learning For Interactive Instruction Following Agentscited by 21
  11. 2024Deep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Bufferscited by 16
  12. 2023Overcoming the Stability Gap in Continual Learningcited by 13
36 more
  1. 2023Kalman Filter for Online Classification of Non-Stationary Datacited by 13
  2. 2023GRASP: A Rehearsal Policy for Efficient Online Continual Learningcited by 13
  3. 2023Continual Learning for Predictive Maintenance: Overview and Challengescited by 10
  4. 2023Cost-effective On-device Continual Learning over Memory Hierarchy with Mirocited by 10
  5. 2022Memory-efficient Reinforcement Learning with Value-based Knowledge Consolidationcited by 9
  6. 2023Few-Shot Open-Set Learning for On-Device Customization of KeyWord Spotting Systemscited by 9
  7. 2023Online Continual Learning for Robust Indoor Object Recognitioncited by 9
  8. 2023Online Continual Learning in Keyword Spotting for Low-Resource Devices via Pooling High-Order Temporal Statisticscited by 9
  9. 2023An Analysis of Initial Training Strategies for Exemplar-Free Class-Incremental Learningcited by 7
  10. 2023Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraintcited by 6
  11. 2024Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learningcited by 6
  12. 2024Activation Map Compression through Tensor Decomposition for Deep Learningcited by 6
  13. 2022Continual-Learning-as-a-Service (CLaaS): On-Demand Efficient Adaptation of Predictive Modelscited by 5
  14. 2024FeTrIL++: Feature Translation for Exemplar-Free Class-Incremental Learning with Hill-Climbingcited by 5
  15. 2024Continual Learning for Autonomous Robots: A Prototype-based Approachcited by 5
  16. 2024How green is continual learning, really? Analyzing the energy consumption in continual training of vision foundation modelscited by 5
  17. 2022Cold Start Streaming Learning for Deep Networkscited by 3
  18. 2023Hadamard Domain Training with Integers for Class Incremental Quantized Learningcited by 3
  19. 2024Swiss DINO: Efficient and Versatile Vision Framework for On-device Personal Object Searchcited by 3
  20. 2023On the challenges to learn from Natural Data Streamscited by 2
  21. 2024REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learningcited by 2
  22. 2025Sufficient conditions for offline reactivation in recurrent neural networkscited by 2
  23. 2025CLoRA: Parameter-Efficient Continual Learning with Low-Rank Adaptationcited by 2
  24. 2023Explainable Lifelong Stream Learning Based on "Glocal" Pairwise Fusioncited by 1
  25. 2023Benchmarking Continual Learning from Cognitive Perspectivescited by 1
  26. 2025A Good Start Matters: Enhancing Continual Learning with Data-Driven Weight Initializationcited by 1
  27. 2026ImageHD: Energy-Efficient On-Device Continual Learning of Visual Representations via Hyperdimensional Computingcited by 1
  28. 2024Recommendation of data-free class-incremental learning algorithms by simulating future datacited by 0
  29. 2024Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learningcited by 0
  30. 2025Generative Binary Memory: Pseudo-Replay Class-Incremental Learning on Binarized Embeddingscited by 0
  31. 2025Responsive DNN Adaptation for Video Analytics against Environment Shift via Hierarchical Mobile-Cloud Collaborationscited by 0
  32. 2025Deep Generative Continual Learning using Functional LoRA: FunLoRAcited by 0
  33. 2025Online Continual Learning on Intel Loihi 2 via a Co-designed Spiking Neural Networkcited by 0
  34. 2026Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edgecited by 0
  35. 2026Scaling few-shot spoken word classification with generative meta-continual learningcited by 0
  36. 2026CLANE: Continual Learning of Actions on Neuromorphic Hardware from Event Camerascited by 0