A Simple and Efficient Stochastic Rounding Method for Training Neural Networks in Low Precision
Not on the map. Why?
Abstract (arXiv)
Conventional stochastic rounding (CSR) is widely employed in the training of neural networks (NNs), showing promising training results even in low-precision computations. We introduce an improved stochastic rounding method, that is simple and efficient. The proposed method succeeds in training NNs with 16-bit fixed-point numbers and provides faster convergence and higher classification accuracy than both CSR and deterministic rounding-to-the-nearest method.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 5 of its 28 references that are in the map. First posted on arXiv 24 Mar 2021.
Builds on
5 of 28 references.
References · 5
- 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
- 2015Deep Learning with Limited Numerical Precisioncited by 368
- 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 163
- 2018Low-Precision Floating-Point Schemes for Neural Network Trainingcited by 4
- 2020Stochastic Rounding: Algorithms and Hardware Acceleratorcited by 4
Built on by
Citing papers per year: 2025: 3.
3 papers.