# Optimizing Tensor Programs on Flexible Storage

arXiv: https://arxiv.org/abs/2210.06267 · HTML: https://mlsanity.com/paper/2210.06267/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2022.

## Built on by (most cited first, 9 of 9)

- [Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity](https://mlsanity.com/paper/2309.10285/) (2023)
- [MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training](https://mlsanity.com/paper/2407.12117/) (2024)
- [XKV: Personalized KV Cache Memory Reduction for Long-Context LLM Inference](https://mlsanity.com/paper/2412.05896/) (2024)
- [Serving Deep Learning Model in Relational Databases](https://mlsanity.com/paper/2310.04696/) (2023)
- [The Duck's Brain: Training and Inference of Neural Networks in Modern Database Engines](https://mlsanity.com/paper/2312.17355/) (2023)
- [Efficient and Sound Differentiable Programming in a Functional Array-Processing Language](https://mlsanity.com/paper/2212.10307/) (2022)
- [nablaSD: Differentiable Programming for Sparse Tensors](https://mlsanity.com/paper/2303.07030/) (2023)
- [Compressing Structured Tensor Algebra](https://mlsanity.com/paper/2407.13726/) (2024)
- [Automated Tensor-Relational Decomposition for Large-Scale Sparse Tensor Computation](https://mlsanity.com/paper/2603.08957/) (2026)

## Abstract (arXiv)

Tensor programs often need to process large tensors (vectors, matrices, or higher order tensors) that require a specialized storage format for their memory layout. Several such layouts have been proposed in the literature, such as the Coordinate Format, the Compressed Sparse Row format, and many others, that were especially designed to optimally store tensors with specific sparsity properties. However, existing tensor processing systems require specialized extensions in order to take advantage of every new storage format. In this paper we describe a system that allows users to define flexible storage formats in a declarative tensor query language, similar to the language used by the tensor program. The programmer only needs to write storage mappings, which describe, in a declarative way, how the tensors are laid out in main memory. Then, we describe a cost-based optimizer that optimizes the tensor program for the specific memory layout. We demonstrate empirically significant performance improvements compared to state-of-the-art tensor processing systems.