mlsanityData to 7 Oct 2026

Dual-side Sparse Tensor Core

arXiv 2105.09564 ↗First posted 20 May 2021cs.AR cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Leveraging sparsity in deep neural network (DNN) models is promising for accelerating model inference. Yet existing GPUs can only leverage the sparsity from weights but not activations, which are dynamic, unpredictable, and hence challenging to exploit. In this work, we propose a novel architecture to efficiently harness the dual-side sparsity (i.e., weight and activation sparsity). We take a systematic approach to understand the (dis)advantages of previous sparsity-related architectures and propose a novel, unexplored paradigm that combines outer-product computation primitive and bitmap-based encoding format. We demonstrate the feasibility of our design with minimal changes to the existing production-scale inner-product-based Tensor Core. We propose a set of novel ISA extensions and co-design the matrix-matrix multiplication and convolution algorithms, which are the two dominant computation patterns in today's DNN models, to exploit our new dual-side sparse Tensor Core. Our evaluation shows that our design can fully unleash the dual-side DNN sparsity and improve the performance by up to one order of magnitude with hlsmall hardware overhead.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 40 of its 75 references that are in the map. First posted on arXiv 20 May 2021.

Builds on

40 of 75 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2017Mask R-CNNcited by 7,259
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  8. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  9. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  10. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 398
  11. 2015LSTM: A Search Space Odysseycited by 315
  12. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
28 more
  1. 2014cuDNN: Efficient Primitives for Deep Learningcited by 240
  2. 2018Efficient Neural Audio Synthesiscited by 218
  3. 2021Accelerating Sparse Deep Neural Networkscited by 213
  4. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
  5. 2016Spatially Adaptive Computation Time for Residual Networkscited by 118
  6. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  7. 2016Quasi-Recurrent Neural Networkscited by 87
  8. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  9. 2018SBNet: Sparse Blocks Network for Fast Inferencecited by 53
  10. 2016ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGAcited by 50
  11. 2020SpArch: Efficient Architecture for Sparse Matrix Multiplicationcited by 47
  12. 2018Deep Learning using Rectified Linear Units (ReLU)cited by 43
  13. 2017NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Mapscited by 39
  14. 2016Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groupscited by 32
  15. 2020Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsitycited by 30
  16. 2017RON: Reverse Connection with Objectness Prior Networks for Object Detectioncited by 27
  17. 2018Balanced Sparsity for Efficient DNN Inference on GPUcited by 27
  18. 2019Adversarial Defense Through Network Profiling Based Path Extractioncited by 22
  19. 2018Efficient Sparse-Winograd Convolutional Neural Networkscited by 20
  20. 2018ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Modelcited by 15
  21. 2019Neural Network Distiller: A Python Package For DNN Compression Researchcited by 12
  22. 2020Ptolemy: Architecture Support for Robust Deep Learningcited by 12
  23. 2018Energy-Constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Maskingcited by 11
  24. 2018Analyzing Machine Learning Workloads Using a Detailed GPU Simulatorcited by 9
  25. 2020How Far Does BERT Look At:Distance-based Clustering and Analysis of BERT's Attentioncited by 9
  26. 2018Modeling Deep Learning Accelerator Enabled GPUscited by 8
  27. 2017Speeding up Convolutional Neural Networks By Exploiting the Sparsity of Rectifier Unitscited by 6
  28. 2020Balancing Efficiency and Flexibility for DNN Acceleration via Temporal GPU-Systolic Array Integrationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 8; 2024: 9; 2025: 3; 2026: 1.

27 papers.

Sort:

By area:

  1. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 51
  2. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  3. 2023TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUscited by 36
  4. 2024A Survey: Collaborative Hardware and Software Design in the Era of Large Language Modelscited by 21
  5. 2025Ditto: Accelerating Diffusion Model via Temporal Value Similaritycited by 18
  6. 2021Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networkscited by 15
  7. 2022Sparseloop: An Analytical Approach To Sparse Tensor Accelerator Modelingcited by 14
  8. 2024vTensor: Flexible Virtual Tensor Management for Efficient LLM Servingcited by 13
  9. 2023HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsitycited by 12
  10. 2024Reconsidering the Energy Efficiency of Spiking Neural Networks Inference from Analytical Perspectivescited by 12
  11. 2024BitMoD: Bit-serial Mixture-of-Datatype LLM Accelerationcited by 12
  12. 2024Enabling Unstructured Sparse Acceleration on Structured Sparse Acceleratorscited by 8
15 more
  1. 2023SGCN: Exploiting Compressed-Sparse Features in Deep Graph Convolutional Network Acceleratorscited by 4
  2. 2023SPADE: Sparse Pillar-based 3D Object Detection Accelerator for Autonomous Drivingcited by 4
  3. 2024Sparse Refinement for Efficient High-Resolution Semantic Segmentationcited by 3
  4. 2024BBS: Bi-directional Bit-level Sparsity for Deep Learning Accelerationcited by 3
  5. 2023CRISP: Hybrid Structured Sparsity for Class-aware Model Pruningcited by 2
  6. 2022Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Trainingcited by 1
  7. 2023Slice-and-Forge: Making Better Use of Caches for Graph Convolutional Network Acceleratorscited by 1
  8. 2023AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUscited by 1
  9. 2025VeriLocc: End-to-End Cross-Architecture Register Allocation via LLMcited by 1
  10. 2021SPA-GCN: Efficient and Flexible GCN Accelerator with an Application for Graph Similarity Computationcited by 0
  11. 2023Rosko: Row Skipping Outer Products for Sparse Matrix Multiplication Kernelscited by 0
  12. 2024FLAASH: Flexible Accelerator Architecture for Sparse High-Order Tensor Contractioncited by 0
  13. 2024Dual sparse training framework: inducing activation map sparsity via Transformed ℓ1 regularizationcited by 0
  14. 2025SparseMap: A Sparse Tensor Accelerator Framework Based on Evolution Strategycited by 0
  15. 2026Celty: SpMSpV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inferencecited by 0