mlsanityData to 7 Oct 2026

Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights

arXiv 2007.00864 ↗First posted 2 Jul 2020cs.AR cs.CV cs.DCcited by 27

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) models are widely used in many important domains. For efficiently processing these computational- and memory-intensive applications, tensors of these over-parameterized models are compressed by leveraging sparsity, size reduction, and quantization of tensors. Unstructured sparsity and tensors with varying dimensions yield irregular computation, communication, and memory access patterns; processing them on hardware accelerators in a conventional manner does not inherently leverage acceleration opportunities. This paper provides a comprehensive survey on the efficient execution of sparse and irregular tensor computations of ML models on hardware accelerators. In particular, it discusses enhancement modules in the architecture design and the software support; categorizes different hardware designs and acceleration techniques and analyzes them in terms of hardware and execution costs; analyzes achievable accelerations for recent DNNs; highlights further opportunities in terms of hardware/software/model co-design optimizations (inter/intra-module). The takeaways from this paper include: understanding the key challenges in accelerating sparse, irregular-shaped, and quantized tensors; understanding enhancements in accelerator systems for supporting their efficient computations; analyzing trade-offs in opting for a specific design choice for encoding, storing, extracting, communicating, computing, and load-balancing the non-zeros; understanding how structured sparsity can improve storage efficiency and balance computations; understanding how to compile and map models with sparse tensors on the accelerators; understanding recent design trends for efficient accelerations and further opportunities.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 4; 2024: 6; 2025: 6; 2026: 2.

27 papers.

Sort:

By area:

  1. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  2. 2022SPDY: Accurate Pruning with Speedup Guaranteescited by 34
  3. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  4. 2025PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Modelscited by 17
  5. 2022Spatial Mixture-of-Expertscited by 14
  6. 2024Training Machine Learning models at the Edge: A Surveycited by 11
  7. 2023Training Large Language Models Efficiently with Sparsity and Dataflowcited by 7
  8. 2022Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systemscited by 6
  9. 2023Sparse-DySta: Sparsity-Aware Dynamic and Static Scheduling for Sparse Multi-DNN Workloadscited by 6
  10. 2025Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detectioncited by 6
  11. 2022AI/ML Algorithms and Applications in VLSI Design and Technologycited by 5
  12. 2022Photonic Reconfigurable Accelerators for Efficient Inference of CNNs with Mixed-Sized Tensorscited by 5
15 more
  1. 2024From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networkscited by 2
  2. 2021A Configurable BNN ASIC using a Network of Programmable Threshold Logic Standard Cellscited by 1
  3. 2022Sparse Periodic Systolic Dataflow for Lowering Latency and Power Dissipation of Convolutional Neural Network Acceleratorscited by 1
  4. 2022LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learningcited by 1
  5. 2023STen: Productive and Efficient Sparsity in PyTorchcited by 1
  6. 2024Combining Neural Architecture Search and Automatic Code Optimization: A Surveycited by 1
  7. 2025OscNet v1.5: Energy Efficient Hopfield Network on CMOS Oscillators for Image Classificationcited by 1
  8. 2024FLAASH: Flexible Accelerator Architecture for Sparse High-Order Tensor Contractioncited by 0
  9. 2024HASS: Hardware-Aware Sparsity Search for Dataflow DNN Acceleratorcited by 0
  10. 2024Efficient Spatio-Temporal Signal Recognition on Edge Devices Using PointLCA-Netcited by 0
  11. 2025MixGCN: Scalable GCN Training by Mixture of Parallelism and Mixture of Acceleratorscited by 0
  12. 2025SpNeRF: Memory Efficient Sparse Volumetric Neural Rendering Accelerator for Edge Devicescited by 0
  13. 2025Data Understanding Survey: Pursuing Improved Dataset Characterization Via Tensor-based Methodscited by 0
  14. 2026Performance and Complexity Trade-off Optimization of Speech Models During Trainingcited by 0
  15. 2026Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compressioncited by 0