# Device-Cloud Collaborative Learning for Recommendation

arXiv: https://arxiv.org/abs/2104.06624 · HTML: https://mlsanity.com/paper/2104.06624/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 26 of 26)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Dynamic Filter Networks](https://mlsanity.com/paper/1605.09673/) (2016)
- [SCAFFOLD: Stochastic Controlled Averaging for Federated Learning](https://mlsanity.com/paper/1910.06378/) (2019)
- [Survey of Personalization Techniques for Federated Learning](https://mlsanity.com/paper/2003.08673/) (2020)
- [JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services](https://mlsanity.com/paper/1801.08618/) (2018)
- [Meta Matrix Factorization for Federated Rating Predictions](https://mlsanity.com/paper/1910.10086/) (2019)
- [Sparse-Interest Network for Sequential Recommendation](https://mlsanity.com/paper/2102.09267/) (2021)
- [On-Device Neural Net Inference with Mobile GPUs](https://mlsanity.com/paper/1907.01989/) (2019)
- [EdgeRec: Recommender System on Edge in Mobile Taobao](https://mlsanity.com/paper/2005.08416/) (2020)
- [A Berkeley View of Systems Challenges for AI](https://mlsanity.com/paper/1712.05855/) (2017)
- [The Matthew effect in empirical data](https://mlsanity.com/paper/1408.5124/) (2014)
- [A Generic Network Compression Framework for Sequential Recommender Systems](https://mlsanity.com/paper/2004.13139/) (2020)
- [Towards Equivalent Transformation of User Preferences in Cross Domain Recommendation](https://mlsanity.com/paper/2009.06884/) (2020)
- [Variational Collaborative Learning for User Probabilistic Representation](https://mlsanity.com/paper/1809.08400/) (2018)

## Built on by (most cited first, 30 of 34)

- [Reliable Adversarial Distillation with Unreliable Teachers](https://mlsanity.com/paper/2106.04928/) (2021)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning](https://mlsanity.com/paper/2205.14833/) (2022)
- [Semi-decentralized Federated Ego Graph Learning for Recommendation](https://mlsanity.com/paper/2302.10900/) (2023)
- [Secure On-Device Video OOD Detection Without Backpropagation](https://mlsanity.com/paper/2503.06166/) (2025)
- [Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world](https://mlsanity.com/paper/2212.00972/) (2022)
- [FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation](https://mlsanity.com/paper/2212.07224/) (2022)
- [Collaboration of Large Language Models and Small Recommendation Models for Device-Cloud Recommendation](https://mlsanity.com/paper/2501.05647/) (2025)
- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Why Do We Click: Visual Impression-aware News Recommendation](https://mlsanity.com/paper/2109.12651/) (2021)
- [Intelligent Request Strategy Design in Recommender System](https://mlsanity.com/paper/2206.12296/) (2022)
- [Device-Cloud Collaborative Recommendation via Meta Controller](https://mlsanity.com/paper/2207.03066/) (2022)
- [Click-through Rate Prediction with Auto-Quantized Contrastive Learning](https://mlsanity.com/paper/2109.13921/) (2021)
- [On-Device Learning with Cloud-Coordinated Data Augmentation for Extreme Model Personalization in Recommender Systems](https://mlsanity.com/paper/2201.10382/) (2022)
- [DC-CCL: Device-Cloud Collaborative Controlled Learning for Large Vision Models](https://mlsanity.com/paper/2303.10361/) (2023)
- [A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Prediction](https://mlsanity.com/paper/2408.09815/) (2024)
- [MC2-SF: Slow-Fast Learning for Mobile-Cloud Collaborative Recommendation](https://mlsanity.com/paper/2109.12314/) (2021)
- [Edge-cloud Collaborative Learning with Federated and Centralized Features](https://mlsanity.com/paper/2304.05871/) (2023)
- [Backpropagation-Free Multi-modal On-Device Model Adaptation via Cloud-Device Collaboration](https://mlsanity.com/paper/2406.01601/) (2024)
- [DIET: Customized Slimming for Incompatible Networks in Sequential Recommendation](https://mlsanity.com/paper/2406.08804/) (2024)
- [Forward Once for All: Structural Parameterized Adaptation for Efficient Cloud-coordinated On-device Recommendation](https://mlsanity.com/paper/2501.02837/) (2025)
- [Socialized Learning: A Survey of the Paradigm Shift for Edge Intelligence in Networked Systems](https://mlsanity.com/paper/2404.13348/) (2024)
- [Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions](https://mlsanity.com/paper/2504.15300/) (2025)
- [Data Leakage via Access Patterns of Sparse Features in Deep Learning-based Recommendation Systems](https://mlsanity.com/paper/2212.06264/) (2022)
- [Mobile Supply: The Last Piece of Jigsaw of Recommender System](https://mlsanity.com/paper/2308.03855/) (2023)
- [ECLM: Efficient Edge-Cloud Collaborative Learning with Continuous Environment Adaptation](https://mlsanity.com/paper/2311.11083/) (2023)
- [Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation](https://mlsanity.com/paper/2409.07416/) (2024)
- [HybridServe: Efficient Serving of Large AI Models with Confidence-Based Cascade Routing](https://mlsanity.com/paper/2505.12566/) (2025)
- [Cloud-Device Collaborative Agents for Sequential Recommendation](https://mlsanity.com/paper/2509.01551/) (2025)

## Abstract (arXiv)

With the rapid development of storage and computing power on mobile devices, it becomes critical and popular to deploy models on devices to save onerous communication latencies and to capture real-time features. While quite a lot of works have explored to facilitate on-device learning and inference, most of them focus on dealing with response delay or privacy protection. Little has been done to model the collaboration between the device and the cloud modeling and benefit both sides jointly. To bridge this gap, we are among the first attempts to study the Device-Cloud Collaborative Learning (DCCL) framework. Specifically, we propose a novel MetaPatch learning approach on the device side to efficiently achieve "thousands of people with thousands of models" given a centralized cloud model. Then, with billions of updated personalized device models, we propose a "model-over-models" distillation algorithm, namely MoMoDistill, to update the centralized cloud model. Our extensive experiments over a range of datasets with different settings demonstrate the effectiveness of such collaboration on both cloud and devices, especially its superiority to model long-tailed users.