# Compressing Structured Tensor Algebra

arXiv: https://arxiv.org/abs/2407.13726 · HTML: https://mlsanity.com/paper/2407.13726/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 60 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [Tensor Decomposition for Signal Processing and Machine Learning](https://mlsanity.com/paper/1607.01668/) (2016)
- [Structured Pruning of Large Language Models](https://mlsanity.com/paper/1910.04732/) (2019)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions](https://mlsanity.com/paper/1802.04730/) (2018)
- [Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code](https://mlsanity.com/paper/1804.10694/) (2018)
- [Tensor Methods in Computer Vision and Deep Learning](https://mlsanity.com/paper/2107.03436/) (2021)
- [SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning](https://mlsanity.com/paper/2207.04606/) (2022)
- [ACDC: A Structured Efficient Linear Layer](https://mlsanity.com/paper/1511.05946/) (2015)
- [Format Abstraction for Sparse Tensor Algebra Compilers](https://mlsanity.com/paper/1804.10112/) (2018)
- [Learning Models over Relational Data using Sparse Tensors and Functional Dependencies](https://mlsanity.com/paper/1703.04780/) (2017)
- [Optimizing Tensor Programs on Flexible Storage](https://mlsanity.com/paper/2210.06267/) (2022)
- [Functional Collection Programming with Semi-Ring Dictionaries](https://mlsanity.com/paper/2103.06376/) (2021)
- [TensorLib: A Spatial Accelerator Generation Framework for Tensor Algebra](https://mlsanity.com/paper/2104.12339/) (2021)
- [Sympiler: Transforming Sparse Matrix Codes by Decoupling Symbolic Analysis](https://mlsanity.com/paper/1705.06575/) (2017)
- [Looplets: A Language For Structured Coiteration](https://mlsanity.com/paper/2209.05250/) (2022)
- [HIR: An MLIR-based Intermediate Representation for Hardware Accelerator Description](https://mlsanity.com/paper/2103.00194/) (2021)

## Abstract (arXiv)

Tensor algebra is a crucial component for data-intensive workloads such as machine learning and scientific computing. As the complexity of data grows, scientists often encounter a dilemma between the highly specialized dense tensor algebra and efficient structure-aware algorithms provided by sparse tensor algebra. In this paper, we introduce DASTAC, a framework to propagate the tensors's captured high-level structure down to low-level code generation by incorporating techniques such as automatic data layout compression, polyhedral analysis, and affine code generation. Our methodology reduces memory footprint by automatically detecting the best data layout, heavily benefits from polyhedral optimizations, leverages further optimizations, and enables parallelization through MLIR. Through extensive experimentation, we show that DASTAC achieves 1 to 2 orders of magnitude speedup over TACO, a state-of-the-art sparse tensor compiler, and StructTensor, a state-of-the-art structured tensor algebra compiler, with a significantly lower memory footprint.