# Acc-SpMM: Accelerating General-purpose Sparse Matrix-Matrix Multiplication with GPU Tensor Cores

arXiv: https://arxiv.org/abs/2501.09251 · HTML: https://mlsanity.com/paper/2501.09251/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2025.

## Built on by (most cited first, 4 of 4)

- [Fused3S: Fast Sparse Attention on Tensor Cores](https://mlsanity.com/paper/2505.08098/) (2025)
- [Libra: Unleashing GPU Heterogeneity for High-Performance Sparse Matrix Multiplication](https://mlsanity.com/paper/2506.22714/) (2025)
- [Unified Static-Dynamic Pruning for Efficient LLM Inference](https://mlsanity.com/paper/2607.21985/) (2026)
- [UnionSparse: An Index-Efficient Sparsity Framework for Low-Bit Sparse LLM Inference on Edge](https://mlsanity.com/paper/2608.09291/) (2026)

## Abstract (arXiv)

General-purpose Sparse Matrix-Matrix Multiplication (SpMM) is a fundamental kernel in scientific computing and deep learning. The emergence of new matrix computation units such as Tensor Cores (TCs) brings more opportunities for SpMM acceleration. However, in order to fully unleash the power of hardware performance, systematic optimization is required. In this paper, we propose Acc-SpMM, a high-performance SpMM library on TCs, with multiple optimizations, including data-affinity-based reordering, memory efficient compressed format, high-throughput pipeline, and adaptive sparsity-aware load balancing. In contrast to the state-of-the-art SpMM kernels on various NVIDIA GPU architectures with a diverse range of benchmark matrices, Acc-SpMM achieves significant performance improvements, on average 2.52x (up to 5.11x) speedup on RTX 4090, on average 1.91x (up to 4.68x) speedup on A800, and on average 1.58x (up to 3.60x) speedup on H100 over cuSPARSE.