Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions
Not on the map. Why?
Abstract (arXiv)
The conventional cloud-based large model learning framework is increasingly constrained by latency, cost, personalization, and privacy concerns. In this survey, we explore an emerging paradigm: collaborative learning between on-device small model and cloud-based large model, which promises low-latency, cost-efficient, and personalized intelligent services while preserving user privacy. We provide a comprehensive review across hardware, system, algorithm, and application layers. At each layer, we summarize key problems and recent advances from both academia and industry. In particular, we categorize collaboration algorithms into data-based, feature-based, and parameter-based frameworks. We also review publicly available datasets and evaluation metrics with user-level or device-level consideration tailored to collaborative learning settings. We further highlight real-world deployments, ranging from recommender systems and mobile livestreaming to personal intelligent assistants. We finally point out open research directions to guide future development in this rapidly evolving field.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 107 of its 160 references that are in the map. First posted on arXiv 17 Apr 2025.
Builds on
107 of 160 references.
References · 107
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2018Federated Optimization in Heterogeneous Networkscited by 2,987
38 more
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2019Searching for MobileNetV3cited by 1,994
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2014FitNets: Hints for Thin Deep Netscited by 1,653
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
- 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
- 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
- 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
- 2019Towards Federated Learning at Scale: System Designcited by 832
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
- 2016Sequence-Level Knowledge Distillationcited by 801
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
- 2018LEAF: A Benchmark for Federated Settingscited by 792
- 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
- 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
- 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
- 2017Federated Multi-Task Learningcited by 672
- 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 652
- 2017Deep Mutual Learningcited by 628
- 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
- 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 541
50 most cited of 107 All 107 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.