# Format Abstraction for Sparse Tensor Algebra Compilers

arXiv: https://arxiv.org/abs/1804.10112 · HTML: https://mlsanity.com/paper/1804.10112/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2018.

## Built on by (most cited first, 16 of 16)

- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding](https://mlsanity.com/paper/2110.10221/) (2021)
- [Sparseloop: An Analytical Approach To Sparse Tensor Accelerator Modeling](https://mlsanity.com/paper/2205.05826/) (2022)
- [JaxPruner: A concise library for sparsity research](https://mlsanity.com/paper/2304.14082/) (2023)
- [BaCO: A Fast and Portable Bayesian Compiler Optimization Framework](https://mlsanity.com/paper/2212.11142/) (2022)
- [Functional Collection Programming with Semi-Ring Dictionaries](https://mlsanity.com/paper/2103.06376/) (2021)
- [UniSparse: An Intermediate Language for General Sparse Format Customization](https://mlsanity.com/paper/2403.05802/) (2024)
- [Scorch: A Library for Sparse Deep Learning](https://mlsanity.com/paper/2405.16883/) (2024)
- [nablaSD: Differentiable Programming for Sparse Tensors](https://mlsanity.com/paper/2303.07030/) (2023)
- [Delta Tensor: Efficient Vector and Tensor Storage in Delta Lake](https://mlsanity.com/paper/2405.03708/) (2024)
- [Compressing Structured Tensor Algebra](https://mlsanity.com/paper/2407.13726/) (2024)
- [FuseFlow: A Fusion-Centric Compilation Framework for Sparse Deep Learning on Streaming Dataflow](https://mlsanity.com/paper/2511.04768/) (2025)
- [Automated Tensor-Relational Decomposition for Large-Scale Sparse Tensor Computation](https://mlsanity.com/paper/2603.08957/) (2026)
- [TensorBench: Benchmarking Coding Agents on a Compiler-Based Tensor Framework](https://mlsanity.com/paper/2606.05570/) (2026)
- [A Hardware-oriented Approach for Efficient Bayesian Inference Computation and Deployment](https://mlsanity.com/paper/2607.17855/) (2026)
- [Efficient Human-Contact Representation for Human-Scene Interaction](https://mlsanity.com/paper/2608.09388/) (2026)

## Abstract (arXiv)

This paper shows how to build a sparse tensor algebra compiler that is agnostic to tensor formats (data layouts). We develop an interface that describes formats in terms of their capabilities and properties, and show how to build a modular code generator where new formats can be added as plugins. We then describe six implementations of the interface that compose to form the dense, CSR/CSF, COO, DIA, ELL, and HASH tensor formats and countless variants thereof. With these implementations at hand, our code generator can generate code to compute any tensor algebra expression on any combination of the aforementioned formats. To demonstrate our technique, we have implemented it in the taco tensor algebra compiler. Our modular code generator design makes it simple to add support for new tensor formats, and the performance of the generated code is competitive with hand-optimized implementations. Furthermore, by extending taco to support a wider range of formats specialized for different application and data characteristics, we can improve end-user application performance. For example, if input data is provided in the COO format, our technique allows computing a single matrix-vector multiplication directly with the data in COO, which is up to 3.6× faster than by first converting the data to CSR.