# SparseDitto: An Agentic Sparse Compilation Framework through Architecture-Aware Synthesis on GPUs

arXiv: https://arxiv.org/abs/2608.05033 · HTML: https://mlsanity.com/paper/2608.05033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 33 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [MaxK-GNN: Extremely Fast GPU Kernel Design for Accelerating Graph Neural Networks Training](https://mlsanity.com/paper/2312.08656/) (2023)
- [QiMeng-Kernel: Macro-Thinking Micro-Coding Paradigm for LLM-Based High-Performance GPU Kernel Generation](https://mlsanity.com/paper/2511.20100/) (2025)
- [TC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs](https://mlsanity.com/paper/2112.02052/) (2021)
- [A unified sparse matrix data format for efficient general sparse matrix-vector multiply on modern processors with wide SIMD units](https://mlsanity.com/paper/1307.6209/) (2013)
- [CSR5: An Efficient Storage Format for Cross-Platform Sparse Matrix-Vector Multiplication](https://mlsanity.com/paper/1503.05032/) (2015)
- [Sextans: A Streaming Accelerator for General-Purpose Sparse-Matrix Dense-Matrix Multiplication](https://mlsanity.com/paper/2109.11081/) (2021)
- [AlphaSparse: Generating High Performance SpMV Codes Directly from Sparse Matrices](https://mlsanity.com/paper/2212.10432/) (2022)
- [CB-SpMV:A Data Aggregating and Balance Algorithm for Cache-Friendly Block-Based SpMV on GPUs](https://mlsanity.com/paper/2605.18515/) (2026)

## Abstract (arXiv)

Sparse matrix computation performance on GPU depends on how representation and execution schedule match the input structure and target hardware. No single implementation consistently dominates across sparsity patterns, operators, and hardwares. Existing sparse compilers and specialized systems cannot cover all of them simultaneously. We present SparseDitto, an agentic sparse compilation framework for sparse matrix computation on GPUs. It jointly synthesizes representation, execution schedule, and hardware mapping in a unified compilation plan. Structural analysis and a learned template-ranking prior guide architecture-aware synthesis. LLM-guided lowering realizes each plan as CUDA code, while target-GPU profiling drives plan refinement. SparseDitto covers multiple operators, e.g., SpMV, SpMM, and SpGEMM, and various representations within one framework. It can also automatically adapt to different hardwares. Across various SuiteSparse matrices, SparseDitto achieves geometric-mean speedups over cuSPARSE of 2.68× on an NVIDIA RTX PRO 6000 and 2.79× on an NVIDIA H200 (up to 146.61×). Its generated SpMM kernels accelerate full-batch GCN training by up to 3.39×.