mlsanityData to 14 Oct 2026

The Ubiquitous Sparse Matrix-Matrix Products

arXiv 2508.04077 ↗First posted 6 Aug 2025math.NA cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Multiplication of a sparse matrix with another (dense or sparse) matrix is a fundamental operation that captures the computational patterns of many data science applications, including but not limited to graph algorithms, sparsely connected neural networks, graph neural networks, clustering, and many-to-many comparisons of biological sequencing data. In many application scenarios, the matrix multiplication takes places on an arbitrary algebraic semiring where the scalar operations are overloaded with user-defined functions with certain properties or a more general heterogenous algebra where even the domains of the input matrices can be different. Here, we provide a unifying treatment of the sparse matrix-matrix operation and its rich application space including machine learning, computational biology and chemistry, graph algorithms, and scientific computing.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 133 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

45 of 133 references.

References · 45

  1. 2017Graph Attention Networkscited by 8,923
  2. 2018How Powerful are Graph Neural Networks?cited by 4,393
  3. 2020Longformer: The Long-Document Transformercited by 3,413
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  5. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 787
  6. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 746
  7. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  8. 2019The State of Sparsity in Deep Neural Networkscited by 476
  9. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 474
  10. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  11. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 354
  12. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
33 more
  1. 2018The Convergence of Sparsified Gradient Methodscited by 229
  2. 2021Accelerating Sparse Deep Neural Networkscited by 223
  3. 2022FP8 Formats for Deep Learningcited by 196
  4. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  5. 2018Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Networkcited by 171
  6. 2020The Hardware Lotterycited by 160
  7. 2020Sparse GPU Kernels for Deep Learningcited by 141
  8. 2022The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformerscited by 122
  9. 2020SpArch: Efficient Architecture for Sparse Matrix Multiplicationcited by 49
  10. 2020GE-SpMM: General-purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networkscited by 42
  11. 2023Randomized Numerical Linear Algebra : A Perspective on the Field With an Eye to Softwarecited by 38
  12. 2020FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systemscited by 36
  13. 2025An Efficient Sparse Kernel Generator for O(3)-Equivariant Deep Networkscited by 22
  14. 2013A convex pseudo-likelihood framework for high dimensional partial correlation estimation with convergence guaranteescited by 12
  15. 2016Scaling betweenness centrality using communication-efficient sparse matrix multiplicationcited by 11
  16. 2019Communication-Efficient Jaccard Similarity for High-Performance Distributed Genome Comparisonscited by 10
  17. 2014Optimization Methods for Sparse Pseudo-Likelihood Graphical Model Selectioncited by 6
  18. 2023Distributed Matrix-Based Sampling for Graph Neural Network Trainingcited by 6
  19. 2017LaraDB: A Minimalist Kernel for Linear and Relational Algebra Computationcited by 5
  20. 2016MPI-FAUN: An MPI-Based Framework for Alternating-Updating Nonnegative Matrix Factorizationcited by 4
  21. 2017Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimationcited by 4
  22. 2015Reducing Parallel Communication in Algebraic Multigrid through Sparsificationcited by 3
  23. 2019On the Dimensionality of Embeddings for Sparse Features and Datacited by 3
  24. 2021GPU Semiring Primitives for Sparse Neighborhood Methodscited by 3
  25. 2021Randomized Algorithms for Scientific Computing (RASC)cited by 3
  26. 2022Distributed-Memory Sparse Kernels for Machine Learningcited by 3
  27. 2020Parallel String Graph Construction and Transitive Reduction for De Novo Genome Assemblycited by 2
  28. 2022High-order Line Graphs of Non-uniform Hypergraphs: Algorithms, Applications, and Experimental Analysiscited by 2
  29. 2022pylspack: Parallel algorithms and data structures for sketching, column subset selection, regression and leverage scorescited by 2
  30. 2025Toward Efficient SpMV in Sparse LLMs via Block Extraction and Compressed Storagecited by 2
  31. 2020Optimizing High Performance Markov Clustering for Pre-Exascale Architecturescited by 1
  32. 2020Distributed Many-to-Many Protein Sequence Alignment using Sparse Matricescited by 1
  33. 2021Enterprise-Scale Search: Accelerating Inference for Sparse Extreme Multi-Label Ranking Treescited by 1

Built on by

Not cited yet.