# TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation

arXiv: https://arxiv.org/abs/2302.05155 · HTML: https://mlsanity.com/paper/2302.05155/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net](https://mlsanity.com/paper/1807.09441/) (2018)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Parameter-free Online Test-time Adaptation](https://mlsanity.com/paper/2201.05718/) (2022)
- [Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data](https://mlsanity.com/paper/2110.03374/) (2021)
- [NOTE: Robust Continual Test-time Adaptation Against Temporal Correlation](https://mlsanity.com/paper/2208.05117/) (2022)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [WildNet: Learning Domain Generalized Semantic Segmentation from the Wild](https://mlsanity.com/paper/2204.01446/) (2022)
- [Revisiting Locally Supervised Learning: an Alternative to End-to-end Training](https://mlsanity.com/paper/2101.10832/) (2021)
- [Improving Test-Time Adaptation via Shift-agnostic Weight Regularization and Nearest Source Prototypes](https://mlsanity.com/paper/2207.11707/) (2022)
- [SITA: Single Image Test-time Adaptation](https://mlsanity.com/paper/2112.02355/) (2021)

## Built on by (most cited first, 30 of 101)

- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization](https://mlsanity.com/paper/2303.01904/) (2023)
- [Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factors](https://mlsanity.com/paper/2403.07366/) (2024)
- [Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering](https://mlsanity.com/paper/2309.17249/) (2023)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [In Search of Lost Online Test-time Adaptation: A Survey](https://mlsanity.com/paper/2310.20199/) (2023)
- [Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization](https://mlsanity.com/paper/2308.06879/) (2023)
- [Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalization](https://mlsanity.com/paper/2309.14949/) (2023)
- [Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization](https://mlsanity.com/paper/2312.10165/) (2023)
- [Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation](https://mlsanity.com/paper/2311.13209/) (2023)
- [BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation](https://mlsanity.com/paper/2402.08712/) (2024)
- [Any-Shift Prompting for Generalization over Distributions](https://mlsanity.com/paper/2402.10099/) (2024)
- [DOTA: Distributional Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2409.19375/) (2024)
- [Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts](https://mlsanity.com/paper/2308.08810/) (2023)
- [Multi-Modal Continual Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2303.10457/) (2023)
- [T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memory](https://mlsanity.com/paper/2403.10052/) (2024)
- [Unified Entropy Optimization for Open-Set Test-Time Adaptation](https://mlsanity.com/paper/2404.06065/) (2024)
- [Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition](https://mlsanity.com/paper/2310.18562/) (2023)
- [Effective Restoration of Source Knowledge in Continual Test Time Adaptation](https://mlsanity.com/paper/2311.04991/) (2023)
- [What, How, and When Should Object Detectors Update in Continually Changing Test Domains?](https://mlsanity.com/paper/2312.08875/) (2023)
- [ATTA: Anomaly-aware Test-Time Adaptation for Out-of-Distribution Detection in Segmentation](https://mlsanity.com/paper/2309.05994/) (2023)
- [Backpropagation-free Network for 3D Test-time Adaptation](https://mlsanity.com/paper/2403.18442/) (2024)
- [STFAR: Improving Object Detection Robustness at Test-Time by Self-Training with Feature Alignment Regularization](https://mlsanity.com/paper/2303.17937/) (2023)
- [Distribution Alignment for Fully Test-Time Adaptation with Dynamic Online Data Streams](https://mlsanity.com/paper/2407.12128/) (2024)
- [CloudFixer: Test-Time Adaptation for 3D Point Clouds via Diffusion-Guided Geometric Transformation](https://mlsanity.com/paper/2407.16193/) (2024)
- [CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation](https://mlsanity.com/paper/2503.18244/) (2025)
- [Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation](https://mlsanity.com/paper/2505.20704/) (2025)
- [Monitoring Risks in Test-Time Adaptation](https://mlsanity.com/paper/2507.08721/) (2025)
- [Unraveling Batch Normalization for Realistic Test-Time Adaptation](https://mlsanity.com/paper/2312.09486/) (2023)

## Abstract (arXiv)

This paper proposes a novel batch normalization strategy for test-time adaptation. Recent test-time adaptation methods heavily rely on the modified batch normalization, i.e., transductive batch normalization (TBN), which calculates the mean and the variance from the current test batch rather than using the running mean and variance obtained from the source data, i.e., conventional batch normalization (CBN). Adopting TBN that employs test batch statistics mitigates the performance degradation caused by the domain shift. However, re-estimating normalization statistics using test data depends on impractical assumptions that a test batch should be large enough and be drawn from i.i.d. stream, and we observed that the previous methods with TBN show critical performance drop without the assumptions. In this paper, we identify that CBN and TBN are in a trade-off relationship and present a new test-time normalization (TTN) method that interpolates the statistics by adjusting the importance between CBN and TBN according to the domain-shift sensitivity of each BN layer. Our proposed TTN improves model robustness to shifted domains across a wide range of batch sizes and in various realistic evaluation scenarios. TTN is widely applicable to other test-time adaptation methods that rely on updating model parameters via backpropagation. We demonstrate that adopting TTN further improves their performance and achieves state-of-the-art performance in various standard benchmarks.