# Bitwidth-Specific Logarithmic Arithmetic for Future Hardware-Accelerated Training

arXiv: https://arxiv.org/abs/2510.17058 · HTML: https://mlsanity.com/paper/2510.17058/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 20 Oct 2025.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [Scalable Methods for 8-bit Training of Neural Networks](https://mlsanity.com/paper/1805.11046/) (2018)
- [Training and Inference with Integers in Deep Neural Networks](https://mlsanity.com/paper/1802.04680/) (2018)
- [NITI: Training Integer Neural Networks Using Integer-only Arithmetic](https://mlsanity.com/paper/2009.13108/) (2020)
- [Is Integer Arithmetic Enough for Deep Learning Training?](https://mlsanity.com/paper/2207.08822/) (2022)
- [Neural Network Training with Approximate Logarithmic Computations](https://mlsanity.com/paper/1910.09876/) (2019)
- [LNS-Madam: Low-Precision Training in Logarithmic Number System using Multiplicative Weight Update](https://mlsanity.com/paper/2106.13914/) (2021)

## Abstract (arXiv)

While advancements in quantization have significantly reduced the computational costs of inference in deep learning, training still predominantly relies on complex floating-point arithmetic. Low-precision fixed-point training presents a compelling alternative. This work introduces a novel enhancement in low-precision logarithmic fixed-point training, geared towards future hardware accelerator designs. We propose incorporating bitwidth in the design of approximations to arithmetic operations. To this end, we introduce a new hardware-friendly, piece-wise linear approximation for logarithmic addition. Using simulated annealing, we optimize this approximation at different precision levels. A C++ bit-true simulation demonstrates training of VGG-11 and VGG-16 models on CIFAR-100 and TinyImageNet, respectively, using 12-bit integer arithmetic with minimal accuracy degradation compared to 32-bit floating-point training. Our hardware study reveals up to 32.5% reduction in area and 53.5% reduction in energy consumption for the proposed LNS multiply-accumulate units compared to that of linear fixed-point equivalents.