mlsanityData to 14 Oct 2026

Balanced Sparsity for Efficient DNN Inference on GPU

arXiv 1811.00206 ↗First posted 1 Nov 2018cs.CVcited by 27

Not on the map. Why?

Abstract (arXiv)

In trained deep neural networks, unstructured pruning can reduce redundant weights to lower storage cost. However, it requires the customization of hardwares to speed up practical inference. Another trend accelerates sparse model inference on general-purpose hardwares by adopting coarse-grained sparsity to prune or regularize consecutive weights for efficient computation. But this method often sacrifices model accuracy. In this paper, we propose a novel fine-grained sparsity approach, balanced sparsity, to achieve high model accuracy with commercial hardwares efficiently. Our approach adapts to high parallelism property of GPU, showing incredible potential for sparsity in the widely deployment of deep learning services. Experiment results show that balanced sparsity achieves up to 3.1x practical speedup for model inference on GPU, while retains the same high model accuracy as fine-grained sparsity.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 3; 2023: 8; 2024: 3; 2025: 2; 2026: 2.

27 papers.

Sort:

By area:

  1. 2021Accelerating Sparse Deep Neural Networkscited by 223
  2. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 209
  3. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  4. 2021Dual-side Sparse Tensor Corecited by 29
  5. 2024Effective Interplay between Sparsity and Quantization: From Theory to Practicecited by 19
  6. 2021Multi-Exit Semantic Segmentation Networkscited by 18
  7. 2024Progressive Gradient Flow for Robust N:M Sparsity Training in Transformerscited by 18
  8. 2024Sparse maximal update parameterization: A holistic approach to sparse training dynamicscited by 16
  9. 2021Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networkscited by 15
  10. 2023E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsitycited by 15
  11. 2021SparseDNN: Fast Sparse Deep Learning Inference on CPUscited by 13
  12. 2022Attentive Fine-Grained Structured Sparsity for Image Restorationcited by 12
15 more
  1. 2022Training Recipe for N:M Structured Sparsity with Decaying Pruning Maskcited by 11
  2. 2023Low-Rank Prune-And-Factorize for Language Model Compressioncited by 11
  3. 2023The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Modelscited by 9
  4. 2026Cut Less, Fold More: Model Compression through the Lens of Projection Geometrycited by 4
  5. 2023PopSparse: Accelerated block sparse matrix multiplication on IPUcited by 3
  6. 2023Pruning Compact ConvNets for Efficient Inferencecited by 2
  7. 2021Graph Conditioned Sparse-Attention for Improved Source Code Understandingcited by 1
  8. 2023Physics-aware Roughness Optimization for Diffractive Optical Neural Networkscited by 1
  9. 2023STen: Productive and Efficient Sparsity in PyTorchcited by 1
  10. 2021Cascade Weight Shedding in Deep Neural Networks: Benefits and Pitfalls for Network Pruningcited by 0
  11. 2021Tight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementationcited by 0
  12. 2023A Generalization of Continuous Relaxation in Structured Pruningcited by 0
  13. 2025NVR: Vector Runahead on NPUs for Sparse Memory Accesscited by 0
  14. 2025Decoding for Punctured Convolutional and Turbo Codes: A Deep Learning Solution for Protocols Compliancecited by 0
  15. 2026A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exitscited by 0