# Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world

arXiv: https://arxiv.org/abs/2212.00972 · HTML: https://mlsanity.com/paper/2212.00972/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 32)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Exploring Visual Prompts for Adapting Large-Scale Models](https://mlsanity.com/paper/2203.17274/) (2022)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [Source-Free Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16372/) (2021)
- [Delving Deep into the Generalization of Vision Transformers under Distribution Shifts](https://mlsanity.com/paper/2106.07617/) (2021)
- [FrameExit: Conditional Early Exiting for Efficient Video Recognition](https://mlsanity.com/paper/2104.13400/) (2021)
- [Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning](https://mlsanity.com/paper/2012.04324/) (2020)
- [Adaptive Class Suppression Loss for Long-Tail Object Detection](https://mlsanity.com/paper/2104.00885/) (2021)
- [Device-Cloud Collaborative Learning for Recommendation](https://mlsanity.com/paper/2104.06624/) (2021)
- [Skip-Convolutions for Efficient Video Processing](https://mlsanity.com/paper/2104.11487/) (2021)
- [Real-Time Video Inference on Edge Devices via Adaptive Model Streaming](https://mlsanity.com/paper/2006.06628/) (2020)
- [On Generalizing Beyond Domains in Cross-Domain Continual Learning](https://mlsanity.com/paper/2203.03970/) (2022)
- [Network Offloading Policies for Cloud Robotics: a Learning-based Approach](https://mlsanity.com/paper/1902.05703/) (2019)
- [Overfitting the Data: Compact Neural Video Delivery via Content-aware Feature Modulation](https://mlsanity.com/paper/2108.08202/) (2021)
- [Barnes-Hut-SNE](https://mlsanity.com/paper/1301.3342/) (2013)

## Built on by (most cited first, 17 of 17)

- [LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding](https://mlsanity.com/paper/2312.14074/) (2023)
- [ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation](https://mlsanity.com/paper/2306.04344/) (2023)
- [Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction](https://mlsanity.com/paper/2303.09792/) (2023)
- [Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data](https://mlsanity.com/paper/2405.06413/) (2024)
- [Cloud-Device Collaborative Learning for Multimodal Large Language Models](https://mlsanity.com/paper/2312.16279/) (2023)
- [Distribution-Aware Continual Test-Time Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2309.13604/) (2023)
- [Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation](https://mlsanity.com/paper/2406.06813/) (2024)
- [Continual Test-Time Adaptation for Object Detection with Adaptive Monitoring and Randomized Restoration](https://mlsanity.com/paper/2406.16439/) (2024)
- [Towards Efficient Object Re-Identification with A Novel Cloud-Edge Collaborative Framework](https://mlsanity.com/paper/2401.02041/) (2024)
- [PM-DETR: Domain Adaptive Prompt Memory for Object Detection with Transformers](https://mlsanity.com/paper/2307.00313/) (2023)
- [UniLCD: Unified Local-Cloud Decision-Making via Reinforcement Learning](https://mlsanity.com/paper/2409.11403/) (2024)
- [High-Quality Pseudo-Label Generation Based on Visual Prompt Assisted Cloud Model Update](https://mlsanity.com/paper/2504.00526/) (2025)
- [Continual Adaptation: Environment-Conditional Parameter Generation for Object Detection in Dynamic Scenarios](https://mlsanity.com/paper/2506.24063/) (2025)
- [Catastrophic Forgetting Mitigation via Discrepancy-Weighted Experience Replay](https://mlsanity.com/paper/2507.00042/) (2025)
- [Tackling Device Data Distribution Real-time Shift via Prototype-based Parameter Editing](https://mlsanity.com/paper/2509.06552/) (2025)
- [Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures](https://mlsanity.com/paper/2604.18133/) (2026)
- [CoVStream: Edge-Cloud Collaboration for Understanding of Long Video Streams](https://mlsanity.com/paper/2606.22804/) (2026)

## Abstract (arXiv)

When facing changing environments in the real world, the lightweight model on client devices suffers from severe performance drops under distribution shifts. The main limitations of the existing device model lie in (1) unable to update due to the computation limit of the device, (2) the limited generalization ability of the lightweight model. Meanwhile, recent large models have shown strong generalization capability on the cloud while they can not be deployed on client devices due to poor computation constraints. To enable the device model to deal with changing environments, we propose a new learning paradigm of Cloud-Device Collaborative Continual Adaptation, which encourages collaboration between cloud and device and improves the generalization of the device model. Based on this paradigm, we further propose an Uncertainty-based Visual Prompt Adapted (U-VPA) teacher-student model to transfer the generalization capability of the large model on the cloud to the device model. Specifically, we first design the Uncertainty Guided Sampling (UGS) to screen out challenging data continuously and transmit the most out-of-distribution samples from the device to the cloud. Then we propose a Visual Prompt Learning Strategy with Uncertainty guided updating (VPLU) to specifically deal with the selected samples with more distribution shifts. We transmit the visual prompts to the device and concatenate them with the incoming data to pull the device testing distribution closer to the cloud training distribution. We conduct extensive experiments on two object detection datasets with continually changing environments. Our proposed U-VPA teacher-student framework outperforms previous state-of-the-art test time adaptation and device-cloud collaboration methods. The code and datasets will be released.