# Compressing Low Precision Deep Neural Networks Using Sparsity-Induced Regularization in Ternary Networks

arXiv: https://arxiv.org/abs/1709.06262 · HTML: https://mlsanity.com/paper/1709.06262/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2017.

## Built on by (most cited first, 1 of 1)

- [Sparsifying Binary Networks](https://mlsanity.com/paper/2207.04974/) (2022)

## Abstract (arXiv)

A low precision deep neural network training technique for producing sparse, ternary neural networks is presented. The technique incorporates hard- ware implementation costs during training to achieve significant model compression for inference. Training involves three stages: network training using L2 regularization and a quantization threshold regularizer, quantization pruning, and finally retraining. Resulting networks achieve improved accuracy, reduced memory footprint and reduced computational complexity compared with conventional methods, on MNIST and CIFAR10 datasets. Our networks are up to 98% sparse and 5 & 11 times smaller than equivalent binary and ternary models, translating to significant resource and speed benefits for hardware implementations.