How Does Batch Normalization Help Binary Training?
Not on the map. Why?
Abstract (arXiv)
Binary Neural Networks (BNNs) are difficult to train, and suffer from drop of accuracy. It appears in practice that BNNs fail to train in the absence of Batch Normalization (BatchNorm) layer. We find the main role of BatchNorm is to avoid exploding gradients in the case of BNNs. This finding suggests that the common initialization methods developed for full-precision networks are irrelevant to BNNs. We build a theoretical study on the role of BatchNorm in binary training, backed up by numerical experiments.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 4; 2022: 2; 2023: 0; 2024: 0; 2025: 0; 2026: 1.
7 papers.
- 2021Enabling Binary Neural Network Training on the Edgecited by 11
- 2021Bit Error Tolerance Metrics for Binarized Neural Networkscited by 2
- 2021Universal Approximation Theorems of Fully Connected Binarized Neural Networkscited by 2
- 2022Lightweight Jet Reconstruction and Identification as an Object Detection Taskcited by 1
- 2021PR-CIM: a Variation-Aware Binary-Neural-Network Framework for Process-Resilient Computation-in-memorycited by 0
- 2022On-Sensor Binarized Fully Convolutional Neural Network with A Pixel Processor Arraycited by 0
- 2026Bringing BNNs to Fast Event Processingcited by 0
- 2026Bringing BNNs to Fast Event Processing7 Oct 2026
- 2022Lightweight Jet Reconstruction and Identification as an Object Detection Task9 Feb 2022
- 2022On-Sensor Binarized Fully Convolutional Neural Network with A Pixel Processor Array2 Feb 2022
- 2021PR-CIM: a Variation-Aware Binary-Neural-Network Framework for Process-Resilient Computation-in-memory19 Oct 2021
- 2021Enabling Binary Neural Network Training on the Edge8 Feb 2021
- 2021Universal Approximation Theorems of Fully Connected Binarized Neural Networks4 Feb 2021
- 2021Bit Error Tolerance Metrics for Binarized Neural Networks2 Feb 2021