# FAT: An In-Memory Accelerator with Fast Addition for Ternary Weight Neural Networks

arXiv: https://arxiv.org/abs/2201.07634 · HTML: https://mlsanity.com/paper/2201.07634/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 66 references that are in the map.
- First posted on arXiv 19 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks](https://mlsanity.com/paper/1708.04485/) (2017)
- [Deep Learning with Low Precision by Half-wave Gaussian Quantization](https://mlsanity.com/paper/1702.00953/) (2017)
- [Towards Unified INT8 Training for Convolutional Neural Network](https://mlsanity.com/paper/1912.12607/) (2019)
- [Very Deep Transformers for Neural Machine Translation](https://mlsanity.com/paper/2008.07772/) (2020)
- [Distribution Adaptive INT8 Quantization for Training CNNs](https://mlsanity.com/paper/2102.04782/) (2021)
- [Least squares binary quantization of neural networks](https://mlsanity.com/paper/2001.02786/) (2020)
- [RTN: Reparameterized Ternary Network](https://mlsanity.com/paper/1912.02057/) (2019)
- [TiM-DNN: Ternary in-Memory accelerator for Deep Neural Networks](https://mlsanity.com/paper/1909.06892/) (2019)
- [Near-Memory Computing: Past, Present, and Future](https://mlsanity.com/paper/1908.02640/) (2019)

## Built on by (most cited first, 2 of 2)

- [TerDiT: Ternary Diffusion Models with Transformers](https://mlsanity.com/paper/2405.14854/) (2024)
- [Towards Efficient In-memory Computing Hardware for Quantized Neural Networks: State-of-the-art, Open Challenges and Perspectives](https://mlsanity.com/paper/2307.03936/) (2023)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) demonstrate excellent performance in various applications but have high computational complexity. Quantization is applied to reduce the latency and storage cost of CNNs. Among the quantization methods, Binary and Ternary Weight Networks (BWNs and TWNs) have a unique advantage over 8-bit and 4-bit quantization. They replace the multiplication operations in CNNs with additions, which are favoured on In-Memory-Computing (IMC) devices. IMC acceleration for BWNs has been widely studied. However, though TWNs have higher accuracy and better sparsity than BWNs, IMC acceleration for TWNs has limited research. TWNs on existing IMC devices are inefficient because the sparsity is not well utilized, and the addition operation is not efficient. In this paper, we propose FAT as a novel IMC accelerator for TWNs. First, we propose a Sparse Addition Control Unit, which utilizes the sparsity of TWNs to skip the null operations on zero weights. Second, we propose a fast addition scheme based on the memory Sense Amplifier to avoid the time overhead of both carry propagation and writing back the carry to memory cells. Third, we further propose a Combined-Stationary data mapping to reduce the data movement of activations and weights and increase the parallelism across memory columns. Simulation results show that for addition operations at the Sense Amplifier level, FAT achieves 2.00X speedup, 1.22X power efficiency, and 1.22X area efficiency compared with a State-Of-The-Art IMC accelerator ParaPIM. FAT achieves 10.02X speedup and 12.19X energy efficiency compared with ParaPIM on networks with 80% average sparsity.