# Load-Balanced Sparse MTTKRP on GPUs

arXiv: https://arxiv.org/abs/1904.03329 · HTML: https://mlsanity.com/paper/1904.03329/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2019.

## Built on by (most cited first, 5 of 5)

- [Fast and Accurate Randomized Algorithms for Low-rank Tensor Decompositions](https://mlsanity.com/paper/2104.01101/) (2021)
- [Reconfigurable Low-latency Memory System for Sparse Matricized Tensor Times Khatri-Rao Product on FPGA](https://mlsanity.com/paper/2109.08874/) (2021)
- [Towards Programmable Memory Controller for Tensor Decomposition](https://mlsanity.com/paper/2207.08298/) (2022)
- [A Sparse Tensor Generator with Efficient Feature Extraction](https://mlsanity.com/paper/2405.04944/) (2024)
- [ReLATE: Accelerating Tensor Decomposition via Safe and Efficient Learning of Sparse Encodings](https://mlsanity.com/paper/2509.00280/) (2025)

## Abstract (arXiv)

Sparse matricized tensor times Khatri-Rao product (MTTKRP) is one of the most computationally expensive kernels in sparse tensor computations. This work focuses on optimizing the MTTKRP operation on GPUs, addressing both performance and storage requirements. We begin by identifying the performance bottlenecks in directly extending the state-of-the-art CSF (compressed sparse fiber) format from CPUs to GPUs. A significant challenge with GPUs compared to multicore CPUs is that of utilizing the much greater degree of parallelism in a load-balanced fashion for irregular computations like sparse MTTKRP. To address this issue, we develop a new storage-efficient representation for tensors that enables high-performance, load-balanced execution of MTTKRP on GPUs. A GPU implementation of sparse MTTKRP using the new sparse tensor representation is shown to outperform all currently known parallel sparse CPU and GPU MTTKRP implementations.