mlsanityData to 14 Oct 2026

Delta: A Cloud-assisted Data Enrichment Framework for On-Device Continual Learning

arXiv 2410.18378 ↗First posted 24 Oct 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

In modern mobile applications, users frequently encounter various new contexts, necessitating on-device continual learning (CL) to ensure consistent model performance. While existing research predominantly focused on developing lightweight CL frameworks, we identify that data scarcity is a critical bottleneck for on-device CL. In this work, we explore the potential of leveraging abundant cloud-side data to enrich scarce on-device data, and propose a private, efficient and effective data enrichment framework Delta. Specifically, Delta first introduces a directory dataset to decompose the data enrichment problem into device-side and cloud-side sub-problems without sharing sensitive data. Next, Delta proposes a soft data matching strategy to effectively solve the device-side sub-problem with sparse user data, and an optimal data sampling scheme for cloud server to retrieve the most suitable dataset for enrichment with low computational complexity. Further, Delta refines the data sampling scheme by jointly considering the impact of enriched data on both new and past contexts, mitigating the catastrophic forgetting issue from a new aspect. Comprehensive experiments across four typical mobile computing tasks with varied data modalities demonstrate that Delta could enhance the overall model accuracy by an average of 15.1%, 12.4%, 1.1% and 5.6% for visual, IMU, audio and textual tasks compared with few-shot CL, and consistently reduce the communication costs by over 90% compared to federated CL.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 39 of its 90 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

39 of 90 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  9. 2016Learning without Forgettingcited by 2,496
  10. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  11. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  12. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
27 more
  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  2. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  3. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  4. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  5. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
  6. 2020Few-Shot Class-Incremental Learningcited by 251
  7. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  8. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 161
  9. 2017A Survey on Homomorphic Encryption Schemes: Theory and Implementationcited by 156
  10. 2022Federated Class-Incremental Learningcited by 127
  11. 2021Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minimacited by 118
  12. 2021Online Coreset Selection for Rehearsal-based Continual Learningcited by 117
  13. 2021A Survey on Data Augmentation for Text Classificationcited by 111
  14. 2022Constrained Few-shot Class-incremental Learningcited by 94
  15. 2014Stochastic Optimization with Importance Samplingcited by 91
  16. 2021Achieving Forgetting Prevention and Knowledge Transfer in Continual Learningcited by 91
  17. 2018Mobile Sensor Data Anonymizationcited by 70
  18. 2021Few-Shot Lifelong Learningcited by 59
  19. 2020Ekya: Continuous Learning of Video Analytics Models on Edge Compute Serverscited by 58
  20. 2022Online Continual Learning for Embedded Devicescited by 49
  21. 2022FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clientscited by 46
  22. 2021A TinyML Platform for On-Device Continual Learning with Quantized Latent Replayscited by 37
  23. 2021Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applicationscited by 17
  24. 2023LifeLearner: Hardware-Aware Meta Continual Learning System for Embedded Computing Platformscited by 11
  25. 2023Cost-effective On-device Continual Learning over Memory Hierarchy with Mirocited by 10
  26. 2023Design Principles for Lifelong Learning AI Acceleratorscited by 9
  27. 2022To Store or Not? Online Data Selection for Federated Learning with Limited Storagecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 1.

6 papers.

  1. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  2. 2025Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directionscited by 2
  3. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  4. 2025Optimizing Storage Overhead of User Behavior Log for ML-embedded Mobile Appscited by 1
  5. 2025SAFA-SNN: Sparsity-Aware On-Device Few-Shot Class-Incremental Learning with Fast-Adaptive Structure of Spiking Neural Networkcited by 0
  6. 2026Optimizing Feature Extraction for On-device Model Inference with User Behavior Sequencescited by 0