mlsanityData to 14 Oct 2026

To Fold or Not to Fold: a Necessary and Sufficient Condition on Batch-Normalization Layers Folding

arXiv 2203.14646 ↗First posted 28 Mar 2022cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Batch-Normalization (BN) layers have become fundamental components in the evermore complex deep neural network architectures. Such models require acceleration processes for deployment on edge devices. However, BN layers add computation bottlenecks due to the sequential operation processing: thus, a key, yet often overlooked component of the acceleration process is BN layers folding. In this paper, we demonstrate that the current BN folding approaches are suboptimal in terms of how many layers can be removed. We therefore provide a necessary and sufficient condition for BN folding and a corresponding optimal algorithm. The proposed approach systematically outperforms existing baselines and allows to dramatically reduce the inference time of deep neural networks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

31 of 39 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  11. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  12. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
19 more
  1. 2016Pyramid Scene Parsing Networkcited by 2,479
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  4. 2018Group Normalizationcited by 1,524
  5. 2019Objects as Pointscited by 1,175
  6. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
  7. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  8. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  9. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  10. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  11. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 240
  12. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  13. 2015Data-free parameter pruning for Deep Neural Networkscited by 112
  14. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  15. 2019Traditional and Heavy-Tailed Self Regularization in Neural Network Modelscited by 105
  16. 2019Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorizationcited by 45
  17. 2019Fighting Quantization Bias With Biascited by 30
  18. 2021RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networkscited by 19
  19. 2019Switchable Normalization for Learning-to-Normalize Deep Representationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 0; 2025: 0; 2026: 1.

4 papers.

  1. 2023PowerQuant: Automorphism Search for Non-Uniform Quantizationcited by 20
  2. 2022SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevancecited by 9
  3. 2023PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansionscited by 0
  4. 2026Towards Deep Encrypted Training: Low-Latency, Memory-Efficient, and High-Throughput Inference for Privacy-Preserving Neural Networkscited by 0