AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments
Not on the map. Why?
Abstract (arXiv)
On-device adapting to continual, unpredictable domain shifts is essential for mobile applications like autonomous driving and augmented reality to deliver seamless user experiences in evolving environments. Test-time adaptation (TTA) emerges as a promising solution by tuning model parameters with unlabeled live data immediately before prediction. However, TTA's unique forward-backward-reforward pipeline notably increases the latency over standard inference, undermining the responsiveness in time-sensitive mobile applications. This paper presents AdaShadow, a responsive test-time adaptation framework for non-stationary mobile data distribution and resource dynamics via selective updates of adaptation-critical layers. Although the tactic is recognized in generic on-device training, TTA's unsupervised and online context presents unique challenges in estimating layer importance and latency, as well as scheduling the optimal layer update plan. AdaShadow addresses these challenges with a backpropagation-free assessor to rapidly identify critical layers, a unit-based runtime predictor to account for resource dynamics in latency estimation, and an online scheduler for prompt layer update planning. Also, AdaShadow incorporates a memory I/O-aware computation reuse scheme to further reduce latency in the reforward pass. Results show that AdaShadow achieves the best accuracy-latency balance under continual shifts. At low memory and energy costs, Adashadow provides a 2x to 3.5x speedup (ms-level) over state-of-the-art TTA methods with comparable accuracy and a 14.8% to 25.4% accuracy boost over efficient supervised methods with similar latency.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 85 references that are in the map. First posted on arXiv 10 Oct 2024.
Builds on
39 of 85 references.
References · 39
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
- 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
- 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
- 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
- 2016Training Deep Nets with Sublinear Memory Costcited by 825
- 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
- 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
27 more
- 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 409
- 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
- 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 235
- 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
- 2017Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learningcited by 192
- 2019Domain Adaptation for Semantic Segmentation with Maximum Squares Losscited by 138
- 2021The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalizationcited by 131
- 2023EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularizationcited by 123
- 2019HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptationcited by 89
- 2019What Would Elsa Do? Freezing Layers During Transformer Fine-Tuningcited by 87
- 2020PyTorch Distributed: Experiences on Accelerating Data Parallel Trainingcited by 87
- 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
- 2019Low-Memory Neural Network Training: A Technical Reportcited by 56
- 2018SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networkscited by 55
- 2017Incremental Adversarial Domain Adaptation for Continually Changing Environmentscited by 52
- 2020Computing Systems for Autonomous Driving: State-of-the-Art and Challengescited by 47
- 2019E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savingscited by 36
- 2018K for the Price of 1: Parameter-efficient Multi-task and Transfer Learningcited by 34
- 2022Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloadingcited by 29
- 2023LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookupcited by 19
- 2023Benchmarking Test-Time Adaptation against Distribution Shifts in Image Classificationcited by 17
- 2023Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognitioncited by 15
- 2023ElasticTrainer: Speeding Up On-Device Training with Runtime Elastic Tensor Selectioncited by 12
- 2022p-Meta: Towards On-device Deep Model Adaptationcited by 8
- 2023Enabling Resource-efficient AIoT System with Cross-level Optimization: A surveycited by 8
- 2022Towards Transmission-Friendly and Robust CNN Models over Cloud and Devicecited by 7
- 2022A New Frontier of AI: On-Device AI Training and Personalizationcited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.