# Tensor Decompositions in Deep Learning

arXiv: https://arxiv.org/abs/2002.11835 · HTML: https://mlsanity.com/paper/2002.11835/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2020.

## Built on by (most cited first, 9 of 9)

- [Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing](https://mlsanity.com/paper/2203.06031/) (2022)
- [Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials](https://mlsanity.com/paper/2211.05274/) (2022)
- [Fast algorithm for overcomplete order-3 tensor decomposition](https://mlsanity.com/paper/2202.06442/) (2022)
- [TQCompressor: improving tensor decomposition methods in neural networks via permutations](https://mlsanity.com/paper/2401.16367/) (2024)
- [BiMLP: Compact Binary Architectures for Vision Multi-Layer Perceptrons](https://mlsanity.com/paper/2212.14158/) (2022)
- [Compressive Sensing with Tensorized Autoencoder](https://mlsanity.com/paper/2303.06235/) (2023)
- [Partial Tensorized Transformers for Natural Language Processing](https://mlsanity.com/paper/2310.20077/) (2023)
- [Enhancing Deep Learning Models through Tensorization: A Comprehensive Survey and Framework](https://mlsanity.com/paper/2309.02428/) (2023)
- [OCTANE -- Optimal Control for Tensor-based Autoencoder Network Emergence: Explicit Case](https://mlsanity.com/paper/2509.08169/) (2025)

## Abstract (arXiv)

The paper surveys the topic of tensor decompositions in modern machine learning applications. It focuses on three active research topics of significant relevance for the community. After a brief review of consolidated works on multi-way data analysis, we consider the use of tensor decompositions in compressing the parameter space of deep learning models. Lastly, we discuss how tensor methods can be leveraged to yield richer adaptive representations of complex data, including structured information. The paper concludes with a discussion on interesting open research challenges.