# Generalized Ternary Connect: End-to-End Learning and Compression of Multiplication-Free Deep Neural Networks

arXiv: https://arxiv.org/abs/1811.04985 · HTML: https://mlsanity.com/paper/1811.04985/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2018.

## Built on by (most cited first, 3 of 3)

- [Quantization-Guided Training for Compact TinyML Models](https://mlsanity.com/paper/2103.06231/) (2021)
- [Real-time Hyper-Dimensional Reconfiguration at the Edge using Hardware Accelerators](https://mlsanity.com/paper/2206.05128/) (2022)
- [Improving the Robustness of Quantized Deep Neural Networks to White-Box Attacks using Stochastic Quantization and Information-Theoretic Ensemble Training](https://mlsanity.com/paper/2312.00105/) (2023)

## Abstract (arXiv)

The use of deep neural networks in edge computing devices hinges on the balance between accuracy and complexity of computations. Ternary Connect (TC) citelin2015neural addresses this issue by restricting the parameters to three levels -1, 0, and +1, thus eliminating multiplications in the forward pass of the network during prediction. We propose Generalized Ternary Connect (GTC), which allows an arbitrary number of levels while at the same time eliminating multiplications by restricting the parameters to integer powers of two. The primary contribution is that GTC learns the number of levels and their values for each layer, jointly with the weights of the network in an end-to-end fashion. Experiments on MNIST and CIFAR-10 show that GTC naturally converges to an `almost binary' network for deep classification networks (e.g. VGG-16) and deep variational auto-encoders, with negligible loss of classification accuracy and comparable visual quality of generated samples respectively. We demonstrate superior compression and similar accuracy of GTC in comparison to several state-of-the-art methods for neural network compression. We conclude with simulations showing the potential benefits of GTC in hardware.