# NITI: Training Integer Neural Networks Using Integer-only Arithmetic

arXiv: https://arxiv.org/abs/2009.13108 · HTML: https://mlsanity.com/paper/2009.13108/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2020.

## Built on by (most cited first, 20 of 20)

- [Is Integer Arithmetic Enough for Deep Learning Training?](https://mlsanity.com/paper/2207.08822/) (2022)
- [Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2505.01043/) (2025)
- [ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimization](https://mlsanity.com/paper/2501.04287/) (2025)
- [PocketNN: Integer-only Training and Inference of Neural Networks via Direct Feedback Alignment and Pocket Activations in Pure C++](https://mlsanity.com/paper/2201.02863/) (2022)
- [On Expressive Power of Quantized Neural Networks under Fixed-Point Arithmetic](https://mlsanity.com/paper/2409.00297/) (2024)
- [Neural Network Training on Encrypted Data with TFHE](https://mlsanity.com/paper/2401.16136/) (2024)
- [HLQ: Fast and Efficient Backpropagation via Hadamard Low-rank Quantization](https://mlsanity.com/paper/2406.15102/) (2024)
- [HOT: Hadamard-based Optimized Training](https://mlsanity.com/paper/2503.21261/) (2025)
- [Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training](https://mlsanity.com/paper/2504.09405/) (2025)
- [An Algebraic View of the Expressivity of Recurrent Language Models](https://mlsanity.com/paper/2606.01765/) (2026)
- [Perturbed Iterate SGD for Lipschitz Continuous Loss Functions with Numerical Error and Adaptive Step Sizes](https://mlsanity.com/paper/2211.04655/) (2022)
- [Lightweight Change Detection in Heterogeneous Remote Sensing Images with Online All-Integer Pruning Training](https://mlsanity.com/paper/2405.01920/) (2024)
- [NITRO-D: Native Integer-only Training of Deep Convolutional Neural Networks](https://mlsanity.com/paper/2407.11698/) (2024)
- [Mathematical Challenges in Deep Learning](https://mlsanity.com/paper/2303.15464/) (2023)
- [PRIOT: Pruning-Based Integer-Only Transfer Learning for Embedded Systems](https://mlsanity.com/paper/2503.16860/) (2025)
- [Bitwidth-Specific Logarithmic Arithmetic for Future Hardware-Accelerated Training](https://mlsanity.com/paper/2510.17058/) (2025)
- [Comprehensive Design Space Exploration for Tensorized Neural Network Hardware Accelerators](https://mlsanity.com/paper/2511.17971/) (2025)
- [AdaHOP: Fast and Accurate Low-Precision Training via Outlier-Pattern-Aware Rotation](https://mlsanity.com/paper/2604.02525/) (2026)
- [Memory-Efficient Integer-Only Training of Deep Neural Networks](https://mlsanity.com/paper/2610.16298/) (2026)
- [LocalTrain: Integer-Only Local Learning for Energy-Efficient CNN Training on Edge FPGAs](https://mlsanity.com/paper/2610.17099/) (2026)

## Abstract (arXiv)

While integer arithmetic has been widely adopted for improved performance in deep quantized neural network inference, training remains a task primarily executed using floating point arithmetic. This is because both high dynamic range and numerical accuracy are central to the success of most modern training algorithms. However, due to its potential for computational, storage and energy advantages in hardware accelerators, neural network training methods that can be implemented with low precision integer-only arithmetic remains an active research challenge. In this paper, we present NITI, an efficient deep neural network training framework that stores all parameters and intermediate values as integers, and computes exclusively with integer arithmetic. A pseudo stochastic rounding scheme that eliminates the need for external random number generation is proposed to facilitate conversion from wider intermediate results to low precision storage. Furthermore, a cross-entropy loss backpropagation scheme computed with integer-only arithmetic is proposed. A proof-of-concept open-source software implementation of NITI that utilizes native 8-bit integer operations in modern GPUs to achieve end-to-end training is presented. When compared with an equivalent training setup implemented with floating point storage and arithmetic, NITI achieves negligible accuracy degradation on the MNIST and CIFAR10 datasets using 8-bit integer storage and computation. On ImageNet, 16-bit integers are needed for weight accumulation with an 8-bit datapath. This achieves training results comparable to all-floating-point implementations.