Effect of Batch Normalization on Noise Resistant Property of Deep Learning Models
Not on the map. Why?
Abstract (arXiv)
The fast execution speed and energy efficiency of analog hardware has made them a strong contender for deployment of deep learning model at the edge. However, there are concerns about the presence of analog noise which causes changes to the weight of the models, leading to performance degradation of deep learning model, despite their inherent noise resistant characteristics. The effect of the popular batch normalization layer on the noise resistant ability of deep learning model is investigated in this work. This systematic study has been carried out by first training different models with and without batch normalization layer on CIFAR10 and CIFAR100 dataset. The weights of the resulting models are then injected with analog noise and the performance of the models on the test dataset is obtained and compared. The results show that the presence of batch normalization layer negatively impacts noise resistant property of deep learning model and the impact grows with the increase of the number of batch normalization layers.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 62 references that are in the map. First posted on arXiv 15 May 2022.
Builds on
30 of 62 references.
References · 30
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2016Layer Normalizationcited by 5,579
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2018Group Normalizationcited by 1,514
- 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
- 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
- 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
- 2018How Does Batch Normalization Help Optimization?cited by 355
- 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
18 more
- 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
- 2018Understanding Batch Normalizationcited by 157
- 2016Improving the Robustness of Deep Neural Networks via Stability Trainingcited by 156
- 2017Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Modelscited by 127
- 2017Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimizationcited by 56
- 2018SmoothOut: Smoothing Out Sharp Minima to Improve Generalization in Deep Learningcited by 25
- 2017Neuromorphic Hardware In The Loop: Training a Deep Spiking Network on the BrainScaleS Wafer-Scale Systemcited by 19
- 2020Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillationcited by 17
- 2017Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Acceleratorcited by 15
- 2018L1-Norm Batch Normalization for Efficient Training of Deep Neural Networkscited by 15
- 2016Deep neural networks are robust to weight binarization and other non-linear distortionscited by 13
- 2021Dynamic Precision Analog Computing for Neural Networkscited by 12
- 2019Improving Noise Tolerance of Mixed-Signal Neural Networkscited by 8
- 2020Benchmarking Inference Performance of Deep Learning Models on Analog Devicescited by 7
- 2020Functional Error Correction for Robust Neural Networkscited by 6
- 2020Device-aware inference operations in SONOS nonvolatile memory arrayscited by 2
- 2018Training Recurrent Neural Networks against Noisy Computations during Inferencecited by 1
- 2018Noisy Computations during Inference: Harmful or Helpful?cited by 1
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.