# Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights

arXiv: https://arxiv.org/abs/2007.00864 · HTML: https://mlsanity.com/paper/2007.00864/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2020.

## Built on by (most cited first, 28 of 28)

- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [SPDY: Accurate Pruning with Speedup Guarantees](https://mlsanity.com/paper/2201.13096/) (2022)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models](https://mlsanity.com/paper/2506.16054/) (2025)
- [Spatial Mixture-of-Experts](https://mlsanity.com/paper/2211.13491/) (2022)
- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Training Large Language Models Efficiently with Sparsity and Dataflow](https://mlsanity.com/paper/2304.05511/) (2023)
- [Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detection](https://mlsanity.com/paper/2504.05411/) (2025)
- [Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems](https://mlsanity.com/paper/2204.09514/) (2022)
- [Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloads](https://mlsanity.com/paper/2310.11096/) (2023)
- [AI/ML Algorithms and Applications in VLSI Design and Technology](https://mlsanity.com/paper/2202.10015/) (2022)
- [Photonic Reconfigurable Accelerators for Efficient Inference of CNNs with Mixed-Sized Tensors](https://mlsanity.com/paper/2207.05278/) (2022)
- [SNE: an Energy-Proportional Digital Accelerator for Sparse Event-Based Convolutions](https://mlsanity.com/paper/2204.10687/) (2022)
- [From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks](https://mlsanity.com/paper/2405.06038/) (2024)
- [A Configurable BNN ASIC using a Network of Programmable Threshold Logic Standard Cells](https://mlsanity.com/paper/2104.01699/) (2021)
- [Sparse Periodic Systolic Dataflow for Lowering Latency and Power Dissipation of Convolutional Neural Network Accelerators](https://mlsanity.com/paper/2207.00068/) (2022)
- [LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.16624/) (2022)
- [STen: Productive and Efficient Sparsity in PyTorch](https://mlsanity.com/paper/2304.07613/) (2023)
- [HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator](https://mlsanity.com/paper/2406.03088/) (2024)
- [Combining Neural Architecture Search and Automatic Code Optimization: A Survey](https://mlsanity.com/paper/2408.04116/) (2024)
- [SpNeRF: Memory Efficient Sparse Volumetric Neural Rendering Accelerator for Edge Devices](https://mlsanity.com/paper/2505.08191/) (2025)
- [OscNet v1.5: Energy Efficient Hopfield Network on CMOS Oscillators for Image Classification](https://mlsanity.com/paper/2506.12610/) (2025)
- [Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compression](https://mlsanity.com/paper/2605.08568/) (2026)
- [FLAASH: Flexible Accelerator Architecture for Sparse High-Order Tensor Contraction](https://mlsanity.com/paper/2404.16317/) (2024)
- [Efficient Spatio-Temporal Signal Recognition on Edge Devices Using PointLCA-Net](https://mlsanity.com/paper/2411.14585/) (2024)
- [MixGCN: Scalable GCN Training by Mixture of Parallelism and Mixture of Accelerators](https://mlsanity.com/paper/2501.01951/) (2025)
- [Data Understanding Survey: Pursuing Improved Dataset Characterization Via Tensor-based Methods](https://mlsanity.com/paper/2510.14161/) (2025)
- [Performance and Complexity Trade-off Optimization of Speech Models During Training](https://mlsanity.com/paper/2601.13704/) (2026)

## Abstract (arXiv)

Machine learning (ML) models are widely used in many important domains. For efficiently processing these computational- and memory-intensive applications, tensors of these over-parameterized models are compressed by leveraging sparsity, size reduction, and quantization of tensors. Unstructured sparsity and tensors with varying dimensions yield irregular computation, communication, and memory access patterns; processing them on hardware accelerators in a conventional manner does not inherently leverage acceleration opportunities. This paper provides a comprehensive survey on the efficient execution of sparse and irregular tensor computations of ML models on hardware accelerators. In particular, it discusses enhancement modules in the architecture design and the software support; categorizes different hardware designs and acceleration techniques and analyzes them in terms of hardware and execution costs; analyzes achievable accelerations for recent DNNs; highlights further opportunities in terms of hardware/software/model co-design optimizations (inter/intra-module). The takeaways from this paper include: understanding the key challenges in accelerating sparse, irregular-shaped, and quantized tensors; understanding enhancements in accelerator systems for supporting their efficient computations; analyzing trade-offs in opting for a specific design choice for encoding, storing, extracting, communicating, computing, and load-balancing the non-zeros; understanding how structured sparsity can improve storage efficiency and balance computations; understanding how to compile and map models with sparse tensors on the accelerators; understanding recent design trends for efficient accelerations and further opportunities.