# SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1708.04485 · HTML: https://mlsanity.com/paper/1708.04485/

## Key facts

- Cited by 135 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2017.

## Built on by (most cited first, 30 of 135)

- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [A Survey of Machine Learning for Computer Architecture and Systems](https://mlsanity.com/paper/2102.07952/) (2021)
- [Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey](https://mlsanity.com/paper/2203.08737/) (2022)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [Multiplying Matrices Without Multiplying](https://mlsanity.com/paper/2106.10860/) (2021)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [Energy-Latency Attacks via Sponge Poisoning](https://mlsanity.com/paper/2203.08147/) (2022)
- [An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers](https://mlsanity.com/paper/2208.06118/) (2022)
- [Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization](https://mlsanity.com/paper/2406.12930/) (2024)
- [Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attention](https://mlsanity.com/paper/2110.09310/) (2021)
- [Exploring Lottery Ticket Hypothesis in Spiking Neural Networks](https://mlsanity.com/paper/2207.01382/) (2022)
- [YOLOPv2: Better, Faster, Stronger for Panoptic Driving Perception](https://mlsanity.com/paper/2208.11434/) (2022)
- [Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Search](https://mlsanity.com/paper/2103.01489/) (2021)
- [Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottlenecks](https://mlsanity.com/paper/2109.14320/) (2021)
- [CoSA: Scheduling by Constrained Optimization for Spatial Accelerators](https://mlsanity.com/paper/2105.01898/) (2021)
- [A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms](https://mlsanity.com/paper/2306.15552/) (2023)
- [S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration](https://mlsanity.com/paper/2107.07983/) (2021)
- [PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization](https://mlsanity.com/paper/2103.01548/) (2021)
- [Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers](https://mlsanity.com/paper/2402.04744/) (2024)
- [Ditto: Accelerating Diffusion Model via Temporal Value Similarity](https://mlsanity.com/paper/2501.11211/) (2025)
- [Sparseloop: An Analytical Approach To Sparse Tensor Accelerator Modeling](https://mlsanity.com/paper/2205.05826/) (2022)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [TwinLiteNet: An Efficient and Lightweight Model for Driveable Area and Lane Segmentation in Self-Driving Cars](https://mlsanity.com/paper/2307.10705/) (2023)
- [Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey](https://mlsanity.com/paper/2205.08099/) (2022)
- [Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputation](https://mlsanity.com/paper/2209.00606/) (2022)
- [HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity](https://mlsanity.com/paper/2305.12718/) (2023)
- [Training Personalized Recommendation Systems from (GPU) Scratch: Look Forward not Backwards](https://mlsanity.com/paper/2205.04702/) (2022)
- [GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent](https://mlsanity.com/paper/2102.07511/) (2021)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) have emerged as a fundamental technology for machine learning. High performance and extreme energy efficiency are critical for deployments of CNNs in a wide range of situations, especially mobile platforms such as autonomous vehicles, cameras, and electronic personal assistants. This paper introduces the Sparse CNN (SCNN) accelerator architecture, which improves performance and energy efficiency by exploiting the zero-valued weights that stem from network pruning during training and zero-valued activations that arise from the common ReLU operator applied during inference. Specifically, SCNN employs a novel dataflow that enables maintaining the sparse weights and activations in a compressed encoding, which eliminates unnecessary data transfers and reduces storage requirements. Furthermore, the SCNN dataflow facilitates efficient delivery of those weights and activations to the multiplier array, where they are extensively reused. In addition, the accumulation of multiplication products are performed in a novel accumulator array. Our results show that on contemporary neural networks, SCNN can improve both performance and energy by a factor of 2.7x and 2.3x, respectively, over a comparably provisioned dense CNN accelerator.