# Split Batch Normalization: Improving Semi-Supervised Learning under Domain Shift

arXiv: https://arxiv.org/abs/1904.03515 · HTML: https://mlsanity.com/paper/1904.03515/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2019.

## Built on by (most cited first, 7 of 7)

- [AugMax: Adversarial Composition of Random Augmentations for Robust Training](https://mlsanity.com/paper/2110.13771/) (2021)
- [A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation](https://mlsanity.com/paper/2104.07256/) (2021)
- [Rethinking "Batch" in BatchNorm](https://mlsanity.com/paper/2105.07576/) (2021)
- [Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity](https://mlsanity.com/paper/2102.11382/) (2021)
- [Separable Batch Normalization for Robust Facial Landmark Localization with Cross-protocol Network Training](https://mlsanity.com/paper/2101.06663/) (2021)
- [Simplifying Multi-Task Architectures Through Task-Specific Normalization](https://mlsanity.com/paper/2512.20420/) (2025)
- [Uncover and Unlearn Nuisances: Agnostic Fully Test-Time Adaptation](https://mlsanity.com/paper/2511.12491/) (2025)

## Abstract (arXiv)

Recent work has shown that using unlabeled data in semi-supervised learning is not always beneficial and can even hurt generalization, especially when there is a class mismatch between the unlabeled and labeled examples. We investigate this phenomenon for image classification on the CIFAR-10 and the ImageNet datasets, and with many other forms of domain shifts applied (e.g. salt-and-pepper noise). Our main contribution is Split Batch Normalization (Split-BN), a technique to improve SSL when the additional unlabeled data comes from a shifted distribution. We achieve it by using separate batch normalization statistics for unlabeled examples. Due to its simplicity, we recommend it as a standard practice. Finally, we analyse how domain shift affects the SSL training process. In particular, we find that during training the statistics of hidden activations in late layers become markedly different between the unlabeled and the labeled examples.