# Neural Network Training with Approximate Logarithmic Computations

arXiv: https://arxiv.org/abs/1910.09876 · HTML: https://mlsanity.com/paper/1910.09876/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [LNS-Madam: Low-Precision Training in Logarithmic Number System using Multiplicative Weight Update](https://mlsanity.com/paper/2106.13914/) (2021)
- [Number Systems for Deep Neural Network Architectures: A Survey](https://mlsanity.com/paper/2307.05035/) (2023)
- [EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices](https://mlsanity.com/paper/2505.02380/) (2025)
- [Approximation Capabilities of Neural Networks using Morphological Perceptrons and Generalizations](https://mlsanity.com/paper/2207.07832/) (2022)
- [Generalized Orders of Magnitude for Scalable, Parallel, High-Dynamic-Range Computation](https://mlsanity.com/paper/2510.03426/) (2025)
- [Bitwidth-Specific Logarithmic Arithmetic for Future Hardware-Accelerated Training](https://mlsanity.com/paper/2510.17058/) (2025)
- [Chameleon: Dynamic Format Adapter for Efficient Diffusion](https://mlsanity.com/paper/2609.33496/) (2026)

## Abstract (arXiv)

The high computational complexity associated with training deep neural networks limits online and real-time training on edge devices. This paper proposed an end-to-end training and inference scheme that eliminates multiplications by approximate operations in the log-domain which has the potential to significantly reduce implementation complexity. We implement the entire training procedure in the log-domain, with fixed-point data representations. This training procedure is inspired by hardware-friendly approximations of log-domain addition which are based on look-up tables and bit-shifts. We show that our 16-bit log-based training can achieve classification accuracy within approximately 1% of the equivalent floating-point baselines for a number of commonly used datasets.