# DELTA: degradation-free fully test-time adaptation

arXiv: https://arxiv.org/abs/2301.13018 · HTML: https://mlsanity.com/paper/2301.13018/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 44 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 30 of 37)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [Maintaining Discrimination and Fairness in Class Incremental Learning](https://mlsanity.com/paper/1911.07053/) (2019)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning](https://mlsanity.com/paper/1702.08811/) (2017)
- [Parameter-free Online Test-time Adaptation](https://mlsanity.com/paper/2201.05718/) (2022)
- [Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation](https://mlsanity.com/paper/2110.04202/) (2021)
- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [Asymmetric Tri-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1702.08400/) (2017)
- [Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models](https://mlsanity.com/paper/1702.03275/) (2017)

## Built on by (most cited first, 30 of 72)

- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [In Search of Lost Online Test-time Adaptation: A Survey](https://mlsanity.com/paper/2310.20199/) (2023)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation](https://mlsanity.com/paper/2311.13209/) (2023)
- [Bayesian Test-Time Adaptation for Vision-Language Models](https://mlsanity.com/paper/2503.09248/) (2025)
- [Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts](https://mlsanity.com/paper/2308.08810/) (2023)
- [Multi-Modal Continual Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2303.10457/) (2023)
- [Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition](https://mlsanity.com/paper/2310.18562/) (2023)
- [Active Test-Time Adaptation: Theoretical Analyses and An Algorithm](https://mlsanity.com/paper/2404.05094/) (2024)
- [What, How, and When Should Object Detectors Update in Continually Changing Test Domains?](https://mlsanity.com/paper/2312.08875/) (2023)
- [Improving Domain Generalization with Domain Relations](https://mlsanity.com/paper/2302.02609/) (2023)
- [Learning to Adapt Frozen CLIP for Few-Shot Test-Time Domain Adaptation](https://mlsanity.com/paper/2506.17307/) (2025)
- [Distribution Alignment for Fully Test-Time Adaptation with Dynamic Online Data Streams](https://mlsanity.com/paper/2407.12128/) (2024)
- [Efficient Test-Time Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2408.05775/) (2024)
- [Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation](https://mlsanity.com/paper/2505.20704/) (2025)
- [Unraveling Batch Normalization for Realistic Test-Time Adaptation](https://mlsanity.com/paper/2312.09486/) (2023)
- [Test-time Adaptation for Regression by Subspace Alignment](https://mlsanity.com/paper/2410.03263/) (2024)
- [SATA: Source Anchoring and Target Alignment Network for Continual Test Time Adaptation](https://mlsanity.com/paper/2304.10113/) (2023)
- [Bag of Tricks for Fully Test-Time Adaptation](https://mlsanity.com/paper/2310.02416/) (2023)
- [Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptation](https://mlsanity.com/paper/2404.10966/) (2024)
- [E-BATS: Efficient Backpropagation-Free Test-Time Adaptation for Speech Foundation Models](https://mlsanity.com/paper/2506.07078/) (2025)
- [Decoupled Prototype Learning for Reliable Test-Time Adaptation](https://mlsanity.com/paper/2401.08703/) (2024)
- [A Layer Selection Approach to Test Time Adaptation](https://mlsanity.com/paper/2404.03784/) (2024)
- [MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection](https://mlsanity.com/paper/2406.14878/) (2024)
- [Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-time Adaptation](https://mlsanity.com/paper/2406.19341/) (2024)
- [Temporal Test-Time Adaptation with State-Space Models](https://mlsanity.com/paper/2407.12492/) (2024)
- [SURGEON: Memory-Adaptive Fully Test-Time Adaptation via Dynamic Activation Sparsity](https://mlsanity.com/paper/2503.20354/) (2025)
- [Self-Bootstrapping for Versatile Test-Time Adaptation](https://mlsanity.com/paper/2504.08010/) (2025)

## Abstract (arXiv)

Fully test-time adaptation aims at adapting a pre-trained model to the test stream during real-time inference, which is urgently required when the test distribution differs from the training distribution. Several efforts have been devoted to improving adaptation performance. However, we find that two unfavorable defects are concealed in the prevalent adaptation methodologies like test-time batch normalization (BN) and self-learning. First, we reveal that the normalization statistics in test-time BN are completely affected by the currently received test samples, resulting in inaccurate estimates. Second, we show that during test-time adaptation, the parameter update is biased towards some dominant classes. In addition to the extensively studied test stream with independent and class-balanced samples, we further observe that the defects can be exacerbated in more complicated test environments, such as (time) dependent or class-imbalanced data. We observe that previous approaches work well in certain scenarios while show performance degradation in others due to their faults. In this paper, we provide a plug-in solution called DELTA for Degradation-freE fuLly Test-time Adaptation, which consists of two components: (i) Test-time Batch Renormalization (TBR), introduced to improve the estimated normalization statistics. (ii) Dynamic Online re-weighTing (DOT), designed to address the class bias within optimization. We investigate various test-time adaptation methods on three commonly used datasets with four scenarios, and a newly introduced real-world dataset. DELTA can help them deal with all scenarios simultaneously, leading to SOTA performance.