mlsanityData to 14 Oct 2026

Group Normalization

arXiv 1803.08494 ↗First posted 22 Mar 2018cs.CV cs.LGcited by 1,524

Not on the map. Why?

Cited by 160 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Batch Normalization (BN) is a milestone technique in the development of deep learning, enabling various networks to train. However, normalizing along the batch dimension introduces problems --- BN's error increases rapidly when the batch size becomes smaller, caused by inaccurate batch statistics estimation. This limits BN's usage for training larger models and transferring features to computer vision tasks including detection, segmentation, and video, which require small batches constrained by memory consumption. In this paper, we present Group Normalization (GN) as a simple alternative to BN. GN divides the channels into groups and computes within each group the mean and variance for normalization. GN's computation is independent of batch sizes, and its accuracy is stable in a wide range of batch sizes. On ResNet-50 trained in ImageNet, GN has 10.6% lower error than its BN counterpart when using a batch size of 2; when using typical batch sizes, GN is comparably good with BN and outperforms other normalization variants. Moreover, GN can be naturally transferred from pre-training to fine-tuning. GN can outperform its BN-based counterparts for object detection and segmentation in COCO, and for video classification in Kinetics, showing that GN can effectively replace the powerful BN in a variety of tasks. GN can be easily implemented by a few lines of code in modern libraries.

Key facts. Cited by 1,524 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 318; 2022: 306; 2023: 279; 2024: 268; 2025: 192; 2026: 161.

1,524 papers.

Sort:

By area:

  1. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  2. 2022A ConvNet for the 2020scited by 3,579
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  4. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  5. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  7. 2022A Generalist Agentcited by 808
  8. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  9. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  10. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 642
  11. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  12. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
38 more
  1. 2021Federated Learning Based on Dynamic Regularizationcited by 562
  2. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  3. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  4. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  5. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  6. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  7. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 471
  8. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  9. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 415
  10. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 379
  11. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 377
  12. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  13. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  14. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
  15. 2021Federated Learning on Non-IID Data: A Surveycited by 346
  16. 2022MAGVIT: Masked Generative Video Transformercited by 346
  17. 2021XCiT: Cross-Covariance Image Transformerscited by 338
  18. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
  19. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
  20. 2021A Field Guide to Federated Optimizationcited by 315
  21. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 309
  22. 2022Perception Prioritized Training of Diffusion Modelscited by 303
  23. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  24. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 294
  25. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 286
  26. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 285
  27. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 266
  28. 2021Mesh Graphormercited by 265
  29. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizercited by 262
  30. 2021Knowledge distillation: A good teacher is patient and consistentcited by 259
  31. 2021Conditional Object-Centric Learning from Videocited by 248
  32. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  33. 2023FuXi: A cascade machine learning forecasting system for 15-day global weather forecastcited by 241
  34. 2021HINet: Half Instance Normalization Network for Image Restorationcited by 237
  35. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  36. 2021Associating Objects with Transformers for Video Object Segmentationcited by 226
  37. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 224
  38. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219

50 most cited of 1,524 All 1,524 (JSON)