mlsanityData to 7 Oct 2026

Enabling Unstructured Sparse Acceleration on Structured Sparse Accelerators

arXiv 2403.07953 ↗First posted 12 Mar 2024cs.LG cs.AI cs.ARcited by 8

Not on the map. Why?

Abstract (arXiv)

Exploiting sparsity in deep neural networks (DNNs) has been a promising area for meeting the growing computation requirements. To minimize the overhead of sparse acceleration, hardware designers have proposed structured sparsity support, but it provides limited flexibility and requires extra model fine-tuning. Moreover, any sparse model fine-tuned for certain structured sparse HW cannot be accelerated by other structured hardware. To enable acceleration using unstructured sparsity of DNNs on structured sparse hardware, we propose an approximation method leveraging the distributive property in linear algebra to turn any sparse tensor into a series of structured sparse tensors. We also develop a software framework, TASDER, to apply high-quality structured approximation on weights and activations of DNNs. Our method accelerates dense and sparse DNNs without fine-tuning and improves energy-delay-product (EDP) by up to 83% and 74%. It achieves up to 39% speed-up on a real system.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 35 of its 67 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

35 of 67 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  6. 2022A ConvNet for the 2020scited by 3,545
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  12. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
23 more
  1. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  2. 2019MLPerf Inference Benchmarkcited by 195
  3. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
  4. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  5. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
  6. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  7. 2021Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Modelscited by 75
  8. 2022SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learningcited by 47
  9. 2017Block-Sparse Recurrent Neural Networkscited by 38
  10. 2017Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networkscited by 36
  11. 2022Learning Best Combination for Efficient N:M Sparsitycited by 32
  12. 2022An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformerscited by 28
  13. 2021Dual-side Sparse Tensor Corecited by 27
  14. 2021CoSA: Scheduling by Constrained Optimization for Spatial Acceleratorscited by 22
  15. 2021S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Accelerationcited by 19
  16. 2020Systolic Tensor Array: An Efficient Structured-Sparse GEMM Accelerator for Mobile CNN Inferencecited by 17
  17. 2024Progressive Gradient Flow for Robust N:M Sparsity Training in Transformerscited by 17
  18. 2022Sparseloop: An Analytical Approach To Sparse Tensor Accelerator Modelingcited by 14
  19. 2023HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsitycited by 12
  20. 2022Training Recipe for N:M Structured Sparsity with Decaying Pruning Maskcited by 10
  21. 2023VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUscited by 8
  22. 2021Doping: A technique for efficient compression of LSTM models using sparse structured additive matricescited by 5
  23. 2021RASA: Efficient Register-Aware Systolic Array Matrix Engine for CPUcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2024Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM modelscited by 12
  2. 2025Accelerating Transformer Inference and Training with 2:4 Activation Sparsitycited by 9
  3. 2025Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systemscited by 0
  4. 2025Leveraging Label Potential for Enhanced Multimodal Emotion Recognitioncited by 0
  5. 2025TriADA: Massively Parallel Trilinear Matrix-by-Tensor Multiply-Add Algorithm and Device Architecture for the Acceleration of 3D Discrete Transformationscited by 0
  6. 2026To 2:4 Sparsity and Beyond: Neuron-level Activation Function to Accelerate LLM Pre-Trainingcited by 0
  7. 2026HieraSparse: Hierarchical Semi-Structured Sparse KV Attentioncited by 0
  8. 2026Different Prompts, Different Ranks: Prompt-aware Dynamic Rank Selection for SVD-based LLM Compressioncited by 0