# S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration

arXiv: https://arxiv.org/abs/2107.07983 · HTML: https://mlsanity.com/paper/2107.07983/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 16 Jul 2021.

## Builds on (most cited first, 24 of 24)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks](https://mlsanity.com/paper/1708.04485/) (2017)
- [ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA](https://mlsanity.com/paper/1612.00694/) (2016)
- [SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers](https://mlsanity.com/paper/1905.12107/) (2019)
- [SpArch: Efficient Architecture for Sparse Matrix Multiplication](https://mlsanity.com/paper/2002.08947/) (2020)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [Bit-pragmatic Deep Neural Network Computing](https://mlsanity.com/paper/1610.06920/) (2016)
- [SCALE-Sim: Systolic CNN Accelerator Simulator](https://mlsanity.com/paper/1811.02883/) (2018)
- [Accelerator-Aware Pruning for Convolutional Neural Networks](https://mlsanity.com/paper/1804.09862/) (2018)
- [Systolic Tensor Array: An Efficient Structured-Sparse GEMM Accelerator for Mobile CNN Inference](https://mlsanity.com/paper/2005.08098/) (2020)
- [TensorDash: Exploiting Sparsity to Accelerate Deep Neural Network Training and Inference](https://mlsanity.com/paper/2009.00748/) (2020)
- [Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization](https://mlsanity.com/paper/1811.04770/) (2018)
- [Euphrates: Algorithm-SoC Co-Design for Low-Power Mobile Continuous Vision](https://mlsanity.com/paper/1803.11232/) (2018)
- [FixyNN: Efficient Hardware for Mobile Computer Vision via Transfer Learning](https://mlsanity.com/paper/1902.11128/) (2019)
- [MASR: A Modular Accelerator for Sparse RNNs](https://mlsanity.com/paper/1908.08976/) (2019)
- [Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices](https://mlsanity.com/paper/2102.07071/) (2021)
- [Mobile Machine Learning Hardware at ARM: A Systems-on-Chip (SoC) Perspective](https://mlsanity.com/paper/1801.06274/) (2018)
- [ASV: Accelerated Stereo Vision System](https://mlsanity.com/paper/1911.07919/) (2019)
- [ISP4ML: Understanding the Role of Image Signal Processing in Efficient Deep Learning Vision Systems](https://mlsanity.com/paper/1911.07954/) (2019)
- [Non-Blocking Simultaneous Multithreading: Embracing the Resiliency of Deep Neural Networks](https://mlsanity.com/paper/2004.09309/) (2020)

## Built on by (most cited first, 19 of 19)

- [Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers](https://mlsanity.com/paper/2402.04744/) (2024)
- [Ditto: Accelerating Diffusion Model via Temporal Value Similarity](https://mlsanity.com/paper/2501.11211/) (2025)
- [PDP: Parameter-free Differentiable Pruning is All You Need](https://mlsanity.com/paper/2305.11203/) (2023)
- [HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity](https://mlsanity.com/paper/2305.12718/) (2023)
- [VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs](https://mlsanity.com/paper/2302.08687/) (2023)
- [Enabling Unstructured Sparse Acceleration on Structured Sparse Accelerators](https://mlsanity.com/paper/2403.07953/) (2024)
- [SDQ: Sparse Decomposed Quantization for LLM Inference](https://mlsanity.com/paper/2406.13868/) (2024)
- [MCBP: A Memory-Compute Efficient LLM Inference Accelerator Leveraging Bit-Slice-enabled Sparsity and Repetitiveness](https://mlsanity.com/paper/2509.10372/) (2025)
- [Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Accelerator](https://mlsanity.com/paper/2504.14365/) (2025)
- [KAN-SAs: Efficient Acceleration of Kolmogorov-Arnold Networks on Systolic Arrays](https://mlsanity.com/paper/2512.00055/) (2025)
- [ELSA: Exploiting Layer-wise N:M Sparsity for Vision Transformer Acceleration](https://mlsanity.com/paper/2409.09708/) (2024)
- [MVQ:Towards Efficient DNN Compression and Acceleration with Masked Vector Quantization](https://mlsanity.com/paper/2412.10261/) (2024)
- [AQPIM: Breaking the PIM Capacity Wall for LLMs with In-Memory Activation Quantization](https://mlsanity.com/paper/2604.18137/) (2026)
- [Periodic Online Testing for Sparse Systolic Tensor Arrays](https://mlsanity.com/paper/2504.18628/) (2025)
- [SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization](https://mlsanity.com/paper/2402.17902/) (2024)
- [VUSA: Virtually Upscaled Systolic Array Architecture to Exploit Unstructured Sparsity in AI Acceleration](https://mlsanity.com/paper/2506.01166/) (2025)
- [SQ-format: A Unified Sparse-Quantized Hardware-friendly Data Format for LLMs](https://mlsanity.com/paper/2512.05409/) (2025)
- [Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlations](https://mlsanity.com/paper/2607.13770/) (2026)
- [HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models](https://mlsanity.com/paper/2607.26515/) (2026)

## Abstract (arXiv)

Exploiting sparsity is a key technique in accelerating quantized convolutional neural network (CNN) inference on mobile devices. Prior sparse CNN accelerators largely exploit un-structured sparsity and achieve significant speedups. Due to the unbounded, largely unpredictable sparsity patterns, however, exploiting unstructured sparsity requires complicated hardware design with significant energy and area overhead, which is particularly detrimental to mobile/IoT inference scenarios where energy and area efficiency are crucial. We propose to exploit structured sparsity, more specifically, Density Bound Block (DBB) sparsity for both weights and activations. DBB block tensors bound the maximum number of non-zeros per block. DBB thus exposes statically predictable sparsity patterns that enable lean sparsity-exploiting hardware. We propose new hardware primitives to implement DBB sparsity for (static) weights and (dynamic) activations, respectively, with very low overheads. Building on top of the primitives, we describe S2TA, a systolic array-based CNN accelerator that exploits joint weight and activation DBB sparsity and new dimensions of data reuse unavailable on the traditional systolic array. S2TA in 16nm achieves more than 2x speedup and energy reduction compared to a strong baseline of a systolic array with zero-value clock gating, over five popular CNN benchmarks. Compared to two recent non-systolic sparse accelerators, Eyeriss v2 (65nm) and SparTen (45nm), S2TA in 65nm uses about 2.2x and 3.1x less energy per inference, respectively.