# Layerwise Noise Maximisation to Train Low-Energy Deep Neural Networks

arXiv: https://arxiv.org/abs/1912.10764 · HTML: https://mlsanity.com/paper/1912.10764/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2019.

## Built on by (most cited first, 3 of 3)

- [MemSE: Fast MSE Prediction for Noisy Memristor-Based DNN Accelerators](https://mlsanity.com/paper/2205.01707/) (2022)
- [Optimizing the Energy Efficiency of Unreliable Memories for Quantized Kalman Filtering](https://mlsanity.com/paper/2109.01520/) (2021)
- [SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness](https://mlsanity.com/paper/2211.11561/) (2022)

## Abstract (arXiv)

Deep neural networks (DNNs) depend on the storage of a large number of parameters, which consumes an important portion of the energy used during inference. This paper considers the case where the energy usage of memory elements can be reduced at the cost of reduced reliability. A training algorithm is proposed to optimize the reliability of the storage separately for each layer of the network, while incurring a negligible complexity overhead compared to a conventional stochastic gradient descent training. For an exponential energy-reliability model, the proposed training approach can decrease the memory energy consumption of a DNN with binary parameters by 3.3× at isoaccuracy, compared to a reliable implementation.