mlsanityData to 7 Oct 2026

Test-time Batch Normalization

arXiv 2205.10210 ↗First posted 20 May 2022cs.LG cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Deep neural networks often suffer the data distribution shift between training and testing, and the batch statistics are observed to reflect the shift. In this paper, targeting of alleviating distribution shift in test time, we revisit the batch normalization (BN) in the training process and reveals two key insights benefiting test-time optimization: (i) preserving the same gradient backpropagation form as training, and (ii) using dataset-level statistics for robust optimization and inference. Based on the two insights, we propose a novel test-time BN layer design, GpreBN, which is optimized during testing by minimizing Entropy loss. We verify the effectiveness of our method on two typical settings with distribution shift, i.e., domain generalization and robustness tasks. Our GpreBN significantly improves the test-time performance and achieves the state of the art results.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 49 of its 64 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

49 of 64 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  7. 2016Wide Residual Networkscited by 3,057
  8. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  9. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  10. 2019Invariant Risk Minimizationcited by 1,832
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  12. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
37 more
  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  2. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  3. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  4. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  5. 2020In Search of Lost Domain Generalizationcited by 1,042
  6. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  7. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  8. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 844
  9. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  10. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  11. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  12. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  13. 2018Recognition in Terra Incognitacited by 655
  14. 2013Domain Generalization via Invariant Feature Representationcited by 622
  15. 2021Domain Generalization with MixStylecited by 563
  16. 2018Generalizing to Unseen Domains via Adversarial Data Augmentationcited by 451
  17. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  18. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  19. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  20. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  21. 2018Two at Once: Enhancing Learning and Generalization Capacities via IBN-Netcited by 361
  22. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 285
  23. 2018Generalizing Across Domains via Cross-Gradient Trainingcited by 268
  24. 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
  25. 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 235
  26. 2021Improving Robustness using Generated Datacited by 233
  27. 2020Improve Unsupervised Domain Adaptation with Mixup Trainingcited by 185
  28. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  29. 2019Learning to Optimize Domain Specific Normalization for Domain Generalizationcited by 160
  30. 2018Domain Generalization via Conditional Invariant Representationcited by 146
  31. 2017A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortionscited by 107
  32. 2020Heterogeneous Domain Generalization via Domain Mixupcited by 103
  33. 2020Learning Invariant Representations and Risks for Semi-supervised Domain Adaptationcited by 69
  34. 2021SITA: Single Image Test-time Adaptationcited by 57
  35. 2020Revisiting Batch Normalization for Improving Corruption Robustnesscited by 55
  36. 2021Test-time Batch Statistics Calibration for Covariate Shiftcited by 50
  37. 2021MixNorm: Test-Time Adaptation Through Online Normalization Estimationcited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 2; 2026: 1.

8 papers.

  1. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  2. 2024Beyond Model Adaptation at Test Time: A Surveycited by 55
  3. 2023In Search of Lost Online Test-time Adaptation: A Surveycited by 43
  4. 2024T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memorycited by 20
  5. 2022Learning Less Generalizable Patterns with an Asymmetrically Trained Double Classifier for Better Test-Time Adaptationcited by 3
  6. 2026Test-Time Adaptation for Non-stationary Time Series: From Synthetic Regime Shifts to Financial Marketscited by 1
  7. 2025Higher-Order Asymptotics of Test-Time Adaptation for Batch Normalization Statisticscited by 0
  8. 2025AutoMiSeg: Automatic Medical Image Segmentation via Test-Time Adaptation of Foundation Modelscited by 0