# Towards Robust and Efficient Cloud-Edge Elastic Model Adaptation via Selective Entropy Distillation arXiv: https://arxiv.org/abs/2402.17316 ยท HTML: https://mlsanity.com/paper/2402.17316/ ## Key facts - Cited by 11 papers in the map (data through 14 Oct 2026). - Builds on 63 of its 85 references that are in the map. - First posted on arXiv 27 Feb 2024. ## Builds on (most cited first, 30 of 63) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015) - [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022) - [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013) - [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019) - [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020) - [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014) - [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018) - [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020) - [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019) - [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020) - [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017) - [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017) - [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017) - [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020) - [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022) - [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022) - [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020) - [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023) - [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019) - [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022) - [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021) - [Semi-supervised Domain Adaptation via Minimax Entropy](https://mlsanity.com/paper/1904.06487/) (2019) - [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021) - [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020) ## Built on by (most cited first, 11 of 11) - [Test-Time Learning for Large Language Models](https://mlsanity.com/paper/2505.20633/) (2025) - [Smoothing the Shift: Towards Stable Test-Time Adaptation under Complex Multimodal Noises](https://mlsanity.com/paper/2503.02616/) (2025) - [Self-Bootstrapping for Versatile Test-Time Adaptation](https://mlsanity.com/paper/2504.08010/) (2025) - [Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs](https://mlsanity.com/paper/2602.07276/) (2026) - [Edge-Cloud Collaborative Motion Planning for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2408.09972/) (2024) - [A Survey on Cloud-Edge-Terminal Collaborative Intelligence in AIoT Networks](https://mlsanity.com/paper/2508.18803/) (2025) - [Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization](https://mlsanity.com/paper/2509.04977/) (2025) - [CPATTA: Conformal Supervision Allocation For Active Test-Time Adaptation](https://mlsanity.com/paper/2509.25692/) (2025) - [ZOTTA: Test-Time Adaptation with Gradient-Free Zeroth-Order Optimization](https://mlsanity.com/paper/2603.14254/) (2026) - [FoCTTA: Low-Memory Continual Test-Time Adaptation with Focus](https://mlsanity.com/paper/2502.20677/) (2025) - [WISE-ATTA: When to Ask for Labels in Budgeted Active Test-Time Adaptation](https://mlsanity.com/paper/2609.37687/) (2026) ## Abstract (arXiv) The conventional deep learning paradigm often involves training a deep model on a server and then deploying the model or its distilled ones to resource-limited edge devices. Usually, the models shall remain fixed once deployed (at least for some period) due to the potential high cost of model adaptation for both the server and edge sides. However, in many real-world scenarios, the test environments may change dynamically (known as distribution shifts), which often results in degraded performance. Thus, one has to adapt the edge models promptly to attain promising performance. Moreover, with the increasing data collected at the edge, this paradigm also fails to further adapt the cloud model for better performance. To address these, we encounter two primary challenges: 1) the edge model has limited computation power and may only support forward propagation; 2) the data transmission budget between cloud and edge devices is limited in latency-sensitive scenarios. In this paper, we establish a Cloud-Edge Elastic Model Adaptation (CEMA) paradigm in which the edge models only need to perform forward propagation and the edge models can be adapted online. In our CEMA, to reduce the communication burden, we devise two criteria to exclude unnecessary samples from uploading to the cloud, i.e., dynamic unreliable and low-informative sample exclusion. Based on the uploaded samples, we update and distribute the affine parameters of normalization layers by distilling from the stronger foundation model to the edge model with a sample replay strategy. Extensive experimental results on ImageNet-C and ImageNet-R verify the effectiveness of our CEMA.