# Higher-Order Asymptotics of Test-Time Adaptation for Batch Normalization Statistics

arXiv: https://arxiv.org/abs/2505.16257 · HTML: https://mlsanity.com/paper/2505.16257/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 19 of 19)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Understanding Batch Normalization](https://mlsanity.com/paper/1806.02375/) (2018)
- [Towards Deeper Graph Neural Networks with Differentiable Group Normalization](https://mlsanity.com/paper/2006.06972/) (2020)
- [TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation](https://mlsanity.com/paper/2302.05155/) (2023)
- [Understanding Test-Time Augmentation](https://mlsanity.com/paper/2402.06892/) (2024)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Test-Time Adaptation Induces Stronger Accuracy and Agreement-on-the-Line](https://mlsanity.com/paper/2310.04941/) (2023)
- [AETTA: Label-Free Accuracy Estimation for Test-Time Adaptation](https://mlsanity.com/paper/2404.01351/) (2024)
- [Test-time Batch Normalization](https://mlsanity.com/paper/2205.10210/) (2022)
- [Information Geometrically Generalized Covariate Shift Adaptation](https://mlsanity.com/paper/2304.09387/) (2023)
- [Test-Time Augmentation Meets Variational Bayes](https://mlsanity.com/paper/2409.12587/) (2024)

## Abstract (arXiv)

This study develops a higher-order asymptotic framework for test-time adaptation (TTA) of Batch Normalization (BN) statistics under distribution shift by integrating classical Edgeworth expansion and saddlepoint approximation techniques with a novel one-step M-estimation perspective. By analyzing the statistical discrepancy between training and test distributions, we derive an Edgeworth expansion for the normalized difference in BN means and obtain an optimal weighting parameter that minimizes the mean-squared error of the adapted statistic. Reinterpreting BN TTA as a one-step M-estimator allows us to derive higher-order local asymptotic normality results, which incorporate skewness and other higher moments into the estimator's behavior. Moreover, we quantify the trade-offs among bias, variance, and skewness in the adaptation process and establish a corresponding generalization bound on the model risk. The refined saddlepoint approximations further deliver uniformly accurate density and tail probability estimates for the BN TTA statistic. These theoretical insights provide a comprehensive understanding of how higher-order corrections and robust one-step updating can enhance the reliability and performance of BN layers in adapting to changing data distributions.