# Mode Normalization

arXiv: https://arxiv.org/abs/1810.05466 · HTML: https://mlsanity.com/paper/1810.05466/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2018.

## Built on by (most cited first, 12 of 12)

- [CrossNorm and SelfNorm for Generalization under Distribution Shifts](https://mlsanity.com/paper/2102.02811/) (2021)
- [Multimodal Federated Learning in Healthcare: a Review](https://mlsanity.com/paper/2310.09650/) (2023)
- [Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence](https://mlsanity.com/paper/2106.03743/) (2021)
- [FedNorm: Modality-Based Normalization in Federated Learning for Multi-Modal Liver Segmentation](https://mlsanity.com/paper/2205.11096/) (2022)
- [Semantic Self-adaptation: Enhancing Generalization with a Single Sample](https://mlsanity.com/paper/2208.05788/) (2022)
- [Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity](https://mlsanity.com/paper/2102.11382/) (2021)
- [Differential privacy for medical deep learning: methods, tradeoffs, and deployment implications](https://mlsanity.com/paper/2506.00660/) (2025)
- [GAN-based Tabular Data Generator for Constructing Synopsis in Approximate Query Processing: Challenges and Solutions](https://mlsanity.com/paper/2212.09015/) (2022)
- [Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filters](https://mlsanity.com/paper/2306.03430/) (2023)
- [A simple normalization technique using window statistics to improve the out-of-distribution generalization on medical images](https://mlsanity.com/paper/2207.03366/) (2022)
- [Supervised Batch Normalization](https://mlsanity.com/paper/2405.17027/) (2024)
- [U-NetMN and SegNetMN: Modified U-Net and SegNet models for bimodal SAR image segmentation](https://mlsanity.com/paper/2506.05444/) (2025)

## Abstract (arXiv)

Normalization methods are a central building block in the deep learning toolbox. They accelerate and stabilize training, while decreasing the dependence on manually tuned learning rate schedules. When learning from multi-modal distributions, the effectiveness of batch normalization (BN), arguably the most prominent normalization method, is reduced. As a remedy, we propose a more flexible approach: by extending the normalization to more than a single mean and variance, we detect modes of data on-the-fly, jointly normalizing samples that share common features. We demonstrate that our method outperforms BN and other widely used normalization techniques in several experiments, including single and multi-task datasets.