# Mixture-of-Rookies: Saving DNN Computations by Predicting ReLU Outputs

arXiv: https://arxiv.org/abs/2202.04990 · HTML: https://mlsanity.com/paper/2202.04990/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 10 Feb 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks](https://mlsanity.com/paper/1708.04485/) (2017)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures](https://mlsanity.com/paper/1911.08460/) (2019)
- [Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure](https://mlsanity.com/paper/1904.03837/) (2019)
- [More is Less: A More Complicated Network with Less Inference Complexity](https://mlsanity.com/paper/1703.08651/) (2017)
- [ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA](https://mlsanity.com/paper/1612.00694/) (2016)
- [PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices](https://mlsanity.com/paper/1909.05073/) (2019)
- [AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates](https://mlsanity.com/paper/1907.03141/) (2019)
- [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://mlsanity.com/paper/1904.02619/) (2019)
- [The High-Dimensional Geometry of Binary Neural Networks](https://mlsanity.com/paper/1705.07199/) (2017)
- [Grow and Prune Compact, Fast, and Accurate LSTMs](https://mlsanity.com/paper/1805.11797/) (2018)
- [PerforatedCNNs: Acceleration through Elimination of Redundant Convolutions](https://mlsanity.com/paper/1504.08362/) (2015)
- [Scaling Up Online Speech Recognition Using ConvNets](https://mlsanity.com/paper/2001.09727/) (2020)
- [PERMDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices](https://mlsanity.com/paper/2004.10936/) (2020)
- [xUnit: Learning a Spatial Activation Function for Efficient Image Restoration](https://mlsanity.com/paper/1711.06445/) (2017)
- [Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computing](https://mlsanity.com/paper/1705.00125/) (2017)
- [MASR: A Modular Accelerator for Sparse RNNs](https://mlsanity.com/paper/1908.08976/) (2019)
- [Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks](https://mlsanity.com/paper/1909.07636/) (2019)
- [Spatial Correlation and Value Prediction in Convolutional Neural Networks](https://mlsanity.com/paper/1807.10598/) (2018)

## Abstract (arXiv)

Deep Neural Networks (DNNs) are widely used in many applications domains. However, they require a vast amount of computations and memory accesses to deliver outstanding accuracy. In this paper, we propose a scheme to predict whether the output of each ReLu activated neuron will be a zero or a positive number in order to skip the computation of those neurons that will likely output a zero. Our predictor, named Mixture-of-Rookies, combines two inexpensive components. The first one exploits the high linear correlation between binarized (1-bit) and full-precision (8-bit) dot products, whereas the second component clusters together neurons that tend to output zero at the same time. We propose a novel clustering scheme based on the analysis of angles, as the sign of the dot product of two vectors depends on the cosine of the angle between them. We implement our hybrid zero output predictor on top of a state-of-the-art DNN accelerator. Experimental results show that our scheme introduces a small area overhead of 5.3% while achieving a speedup of 1.2x and reducing energy consumption by 16.5% on average for a set of diverse DNNs.