mlsanityData to 13 Oct 2026

CrossNorm and SelfNorm for Generalization under Distribution Shifts

arXiv 2102.02811 ↗First posted 4 Feb 2021cs.CV cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Traditional normalization techniques (e.g., Batch Normalization and Instance Normalization) generally and simplistically assume that training and test data follow the same distribution. As distribution shifts are inevitable in real-world applications, well-trained models with previous normalization methods can perform badly in new environments. Can we develop new normalization methods to improve generalization robustness under distribution shifts? In this paper, we answer the question by proposing CrossNorm and SelfNorm. CrossNorm exchanges channel-wise mean and variance between feature maps to enlarge training distribution, while SelfNorm uses attention to recalibrate the statistics to bridge gaps between training and test distributions. CrossNorm and SelfNorm can complement each other, though exploring different directions in statistics usage. Extensive experiments on different fields (vision and language), tasks (classification and segmentation), settings (supervised and semi-supervised), and distribution shift types (synthetic and natural) show the effectiveness. Code is available at https://github.com/amazon-research/crossnorm-selfnorm

Key facts. Cited by 33 papers in the map (data through 13 Oct 2026). Builds on 44 of its 55 references that are in the map. First posted on arXiv 4 Feb 2021.

Builds on

44 of 55 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,991
  4. 2016Densely Connected Convolutional Networkscited by 6,707
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
  6. 2016Layer Normalizationcited by 5,624
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,170
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,849
  9. 2017Squeeze-and-Excitation Networkscited by 4,772
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,435
  11. 2016Wide Residual Networkscited by 3,063
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,882
32 more
  1. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,862
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,544
  3. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,266
  4. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,082
  5. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,005
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,847
  7. 2018Group Normalizationcited by 1,518
  8. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,510
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  10. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,193
  11. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,126
  12. 2014Striving for Simplicity: The All Convolutional Netcited by 989
  13. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 899
  14. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  15. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  16. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 429
  17. 2018Two at Once: Enhancing Learning and Generalization Capacities via IBN-Netcited by 361
  18. 2017Universal Style Transfer via Feature Transformscited by 353
  19. 2016A Learned Representation For Artistic Stylecited by 340
  20. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 294
  21. 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 286
  22. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 245
  23. 2019Fast AutoAugmentcited by 243
  24. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  25. 2019Population Based Augmentation: Efficient Learning of Augmentation Policy Schedulescited by 153
  26. 2019Do Image Classifiers Generalize Across Time?cited by 59
  27. 2019Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noisecited by 28
  28. 2019Using Videos to Evaluate Image Model Robustnesscited by 13
  29. 2019Attentive Normalizationcited by 13
  30. 2018Mode Normalizationcited by 12
  31. 2020Exemplar Normalization for Learning Deep Representationcited by 6
  32. 2020ObjectNet Dataset: Reanalysis and Correctioncited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 1; 2022: 12; 2023: 9; 2024: 4; 2025: 4; 2026: 3.

33 papers.

Sort:

By area:

  1. 2022Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalizationcited by 122
  2. 2022Visual Prompt Tuning for Test-time Domain Adaptationcited by 97
  3. 2023Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentationcited by 82
  4. 2022Pin the Memory: Learning to Generalize Semantic Segmentationcited by 48
  5. 2021Neural Mean Discrepancy for Efficient Out-of-Distribution Detectioncited by 42
  6. 2024SoMA: Singular Value Decomposed Minor Components Adaptation for Domain Generalizable Representation Learningcited by 22
  7. 2023Normalization Enhances Generalization in Visual Reinforcement Learningcited by 13
  8. 2022Semantic Self-adaptation: Enhancing Generalization with a Single Samplecited by 8
  9. 2022Normalization Perturbation: A Simple Domain Generalization Method for Real-World Domain Shiftscited by 8
  10. 2023Moderately Distributional Exploration for Domain Generalizationcited by 8
  11. 2024LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalizationcited by 8
  12. 2024Generalized Face Anti-spoofing via Finer Domain Partition and Disentangling Liveness-irrelevant Factorscited by 6
21 more
  1. 2025Stronger, Steadier & Superior: Geometric Consistency in Depth VFM Forges Domain Generalized Semantic Segmentationcited by 6
  2. 2022Cross-Domain Few-Shot Classification via Inter-Source Stylizationcited by 4
  3. 2023INSURE: An Information Theory Inspired Disentanglement and Purification Model for Domain Generalizationcited by 4
  4. 2022Cooperative Self-Training for Multi-Target Adaptive Semantic Segmentationcited by 3
  5. 2023A Novel Cross-Perturbation for Single Domain Generalizationcited by 3
  6. 2022Bridging the Source-to-target Gap for Cross-domain Person Re-Identification with Intermediate Domainscited by 2
  7. 2022Style Variable and Irrelevant Learning for Generalizable Person Re-identificationcited by 2
  8. 2022Learning Gradient-based Mixup with Extrapolation toward Flatter Minima for Domain Generalizationcited by 2
  9. 2023Learning to Generalize towards Unseen Domains via a Content-Aware Style Invariant Model for Disease Detection from Chest X-rayscited by 2
  10. 2024Domain Expansion and Boundary Growth for Open-Set Single-Source Domain Generalizationcited by 2
  11. 2025Out-of-Distribution Generalization on Graphs via Progressive Inferencecited by 2
  12. 2025Rein++: Efficient Generalization and Adaptation for Semantic Segmentation with Vision Foundation Modelscited by 2
  13. 2026Mode-Dependent Rectification for Stable PPO Trainingcited by 1
  14. 2022A simple normalization technique using window statistics to improve the out-of-distribution generalization on medical imagescited by 0
  15. 2022A Short Survey of Systematic Generalizationcited by 0
  16. 2023Patch-aware Batch Normalization for Improving Cross-domain Robustnesscited by 0
  17. 2023Shape Guided Gradient Voting for Domain Generalizationcited by 0
  18. 2023Boosting Generalization with Adaptive Style Techniques for Fingerprint Liveness Detectioncited by 0
  19. 2025EasyRobust: A Comprehensive and Easy-to-use Toolkit for Robust and Generalized Visioncited by 0
  20. 2026FedBPrompt: Federated Domain Generalization Person Re-Identification via Body Distribution Aware Visual Promptscited by 0
  21. 2026CO-EVO: Co-evolving Semantic Anchoring and Style Diversification for Federated DG-ReIDcited by 0