# UniSparse: An Intermediate Language for General Sparse Format Customization

arXiv: https://arxiv.org/abs/2403.05802 · HTML: https://mlsanity.com/paper/2403.05802/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 37 references that are in the map.
- First posted on arXiv 9 Mar 2024.

## Builds on (most cited first, 10 of 10)

- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [MLIR: A Compiler Infrastructure for the End of Moore's Law](https://mlsanity.com/paper/2002.11054/) (2020)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [Format Abstraction for Sparse Tensor Algebra Compilers](https://mlsanity.com/paper/1804.10112/) (2018)
- [Compiler Support for Sparse Tensor Computations in MLIR](https://mlsanity.com/paper/2202.04305/) (2022)
- [Sextans: A Streaming Accelerator for General-Purpose Sparse-Matrix Dense-Matrix Multiplication](https://mlsanity.com/paper/2109.11081/) (2021)
- [Automatic Generation of Efficient Sparse Tensor Format Conversion Routines](https://mlsanity.com/paper/2001.02609/) (2020)
- [Serpens: A High Bandwidth Memory Based Accelerator for General-Purpose Sparse Matrix-Vector Multiplication](https://mlsanity.com/paper/2111.12555/) (2021)
- [Accelerating the SpMV kernel on standard CPUs by exploiting the partially diagonal structures](https://mlsanity.com/paper/2105.04937/) (2021)

## Built on by (most cited first, 4 of 4)

- [Libra: Unleashing GPU Heterogeneity for High-Performance Sparse Matrix Multiplication](https://mlsanity.com/paper/2506.22714/) (2025)
- [Ember: A Compiler for Efficient Embedding Operations on Decoupled Access-Execute Architectures](https://mlsanity.com/paper/2504.09870/) (2025)
- [Morphing-based Compression for Data-centric ML Pipelines](https://mlsanity.com/paper/2504.11067/) (2025)
- [FuseFlow: A Fusion-Centric Compilation Framework for Sparse Deep Learning on Streaming Dataflow](https://mlsanity.com/paper/2511.04768/) (2025)

## Abstract (arXiv)

The ongoing trend of hardware specialization has led to a growing use of custom data formats when processing sparse workloads, which are typically memory-bound. These formats facilitate optimized software/hardware implementations by utilizing sparsity pattern- or target-aware data structures and layouts to enhance memory access latency and bandwidth utilization. However, existing sparse tensor programming models and compilers offer little or no support for productively customizing the sparse formats. Additionally, because these frameworks represent formats using a limited set of per-dimension attributes, they lack the flexibility to accommodate numerous new variations of custom sparse data structures and layouts. To overcome this deficiency, we propose UniSparse, an intermediate language that provides a unified abstraction for representing and customizing sparse formats. Unlike the existing attribute-based frameworks, UniSparse decouples the logical representation of the sparse tensor (i.e., the data structure) from its low-level memory layout, enabling the customization of both. As a result, a rich set of format customizations can be succinctly expressed in a small set of well-defined query, mutation, and layout primitives. We also develop a compiler leveraging the MLIR infrastructure, which supports adaptive customization of formats, and automatic code generation of format conversion and compute operations for heterogeneous architectures. We demonstrate the efficacy of our approach through experiments running commonly-used sparse linear algebra operations with specialized formats on multiple different hardware targets, including an Intel CPU, an NVIDIA GPU, an AMD Xilinx FPGA, and a simulated processing-in-memory (PIM) device.