# MixNorm: Test-Time Adaptation Through Online Normalization Estimation

arXiv: https://arxiv.org/abs/2110.11478 · HTML: https://mlsanity.com/paper/2110.11478/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 21 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [First-Generation Inference Accelerator Deployment at Facebook](https://mlsanity.com/paper/2107.04140/) (2021)

## Built on by (most cited first, 30 of 47)

- [Continual Test-Time Domain Adaptation](https://mlsanity.com/paper/2203.13591/) (2022)
- [Towards Stable Test-Time Adaptation in Dynamic Wild World](https://mlsanity.com/paper/2302.12400/) (2023)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization](https://mlsanity.com/paper/2303.01904/) (2023)
- [TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation](https://mlsanity.com/paper/2302.05155/) (2023)
- [Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering](https://mlsanity.com/paper/2309.17249/) (2023)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [SITA: Single Image Test-time Adaptation](https://mlsanity.com/paper/2112.02355/) (2021)
- [Beyond Model Adaptation at Test Time: A Survey](https://mlsanity.com/paper/2411.03687/) (2024)
- [In Search of Lost Online Test-time Adaptation: A Survey](https://mlsanity.com/paper/2310.20199/) (2023)
- [Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization](https://mlsanity.com/paper/2312.10165/) (2023)
- [Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approach](https://mlsanity.com/paper/2408.07511/) (2024)
- [CAFA: Class-Aware Feature Alignment for Test-Time Adaptation](https://mlsanity.com/paper/2206.00205/) (2022)
- [Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition](https://mlsanity.com/paper/2203.14222/) (2022)
- [Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition](https://mlsanity.com/paper/2310.18562/) (2023)
- [ATTA: Anomaly-aware Test-Time Adaptation for Out-of-Distribution Detection in Segmentation](https://mlsanity.com/paper/2309.05994/) (2023)
- [Learning Instance-Specific Adaptation for Cross-Domain Segmentation](https://mlsanity.com/paper/2203.16530/) (2022)
- [Uncovering Adversarial Risks of Test-Time Adaptation](https://mlsanity.com/paper/2301.12576/) (2023)
- [SCAP: Transductive Test-Time Adaptation via Supportive Clique-based Attribute Prompting](https://mlsanity.com/paper/2503.12866/) (2025)
- [Online Test-Time Adaptation of Spatial-Temporal Traffic Flow Forecasting](https://mlsanity.com/paper/2401.04148/) (2024)
- [Towards Robust and Efficient Cloud-Edge Elastic Model Adaptation via Selective Entropy Distillation](https://mlsanity.com/paper/2402.17316/) (2024)
- [Beyond Entropy: Region Confidence Proxy for Wild Test-Time Adaptation](https://mlsanity.com/paper/2505.20704/) (2025)
- [Adaptive Domain Generalization via Online Disagreement Minimization](https://mlsanity.com/paper/2208.01996/) (2022)
- [Test-time Batch Normalization](https://mlsanity.com/paper/2205.10210/) (2022)
- [Un-Mixing Test-Time Normalization Statistics: Combatting Label Temporal Correlation](https://mlsanity.com/paper/2401.08328/) (2024)
- [Decoupled Prototype Learning for Reliable Test-Time Adaptation](https://mlsanity.com/paper/2401.08703/) (2024)
- [Self-Bootstrapping for Versatile Test-Time Adaptation](https://mlsanity.com/paper/2504.08010/) (2025)
- [A Simple Test-Time Method for Out-of-Distribution Detection](https://mlsanity.com/paper/2207.08210/) (2022)
- [Towards Unified and Effective Domain Generalization](https://mlsanity.com/paper/2310.10008/) (2023)
- [Continual Test-Time Adaptation for Object Detection with Adaptive Monitoring and Randomized Restoration](https://mlsanity.com/paper/2406.16439/) (2024)

## Abstract (arXiv)

We present a simple and effective way to estimate the batch-norm statistics during test time, to fast adapt a source model to target test samples. Known as Test-Time Adaptation, most prior works studying this task follow two assumptions in their evaluation where (1) test samples come together as a large batch, and (2) all from a single test distribution. However, in practice, these two assumptions may not stand, the reasons for which we propose two new evaluation settings where batch sizes are arbitrary and multiple distributions are considered. Unlike the previous methods that require a large batch of single distribution during test time to calculate stable batch-norm statistics, our method avoid any dependency on large online batches and is able to estimate accurate batch-norm statistics with a single sample. The proposed method significantly outperforms the State-Of-The-Art in the newly proposed settings in Test-Time Adaptation Task, and also demonstrates improvements in various other settings such as Source-Free Unsupervised Domain Adaptation and Zero-Shot Classification.