# Introduction to Tensor Decompositions and their Applications in Machine Learning

arXiv: https://arxiv.org/abs/1711.10781 · HTML: https://mlsanity.com/paper/1711.10781/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2017.

## Built on by (most cited first, 30 of 46)

- [FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2108.06098/) (2021)
- [Learning Frequency Domain Approximation for Binary Neural Networks](https://mlsanity.com/paper/2103.00841/) (2021)
- [Estimator Meets Equilibrium Perspective: A Rectified Straight Through Estimator for Binary Neural Networks Training](https://mlsanity.com/paper/2308.06689/) (2023)
- [Factor Graph Neural Networks](https://mlsanity.com/paper/2308.00887/) (2023)
- [CoordX: Accelerating Implicit Neural Representation with a Split MLP Architecture](https://mlsanity.com/paper/2201.12425/) (2022)
- [Robust Tensor CUR Decompositions: Rapid Low-Tucker-Rank Tensor Recovery with Sparse Corruption](https://mlsanity.com/paper/2305.04080/) (2023)
- [When Random Tensors meet Random Matrices](https://mlsanity.com/paper/2112.12348/) (2021)
- [Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing](https://mlsanity.com/paper/2203.06031/) (2022)
- [Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs](https://mlsanity.com/paper/2205.00484/) (2022)
- [Faster Robust Tensor Power Method for Arbitrary Order](https://mlsanity.com/paper/2306.00406/) (2023)
- [Tensor Logic: The Language of AI](https://mlsanity.com/paper/2510.12269/) (2025)
- [STNN-DDI: A Substructure-aware Tensor Neural Network to Predict Drug-Drug Interactions](https://mlsanity.com/paper/2111.05708/) (2021)
- [Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars](https://mlsanity.com/paper/2212.09140/) (2022)
- [Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression](https://mlsanity.com/paper/2405.12591/) (2024)
- [Support Recovery of Sparse Signals from a Mixture of Linear Measurements](https://mlsanity.com/paper/2106.05951/) (2021)
- [Subquadratic Kronecker Regression with Applications to Tensor Decomposition](https://mlsanity.com/paper/2209.04876/) (2022)
- [Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials](https://mlsanity.com/paper/2211.05274/) (2022)
- [Tensor decomposition for learning Gaussian mixtures from moments](https://mlsanity.com/paper/2106.00555/) (2021)
- [Fast Low-Rank Tensor Decomposition by Ridge Leverage Score Sampling](https://mlsanity.com/paper/2107.10654/) (2021)
- [Fast algorithm for overcomplete order-3 tensor decomposition](https://mlsanity.com/paper/2202.06442/) (2022)
- [Optimizing Orthogonalized Tensor Deflation via Random Tensor Theory](https://mlsanity.com/paper/2302.05798/) (2023)
- [SetConv: A New Approach for Learning from Imbalanced Data](https://mlsanity.com/paper/2104.06313/) (2021)
- [Singular Value Representation: A New Graph Perspective On Neural Networks](https://mlsanity.com/paper/2302.08183/) (2023)
- [A Random Matrix Approach to Low-Multilinear-Rank Tensor Approximation](https://mlsanity.com/paper/2402.03169/) (2024)
- [Tensor learning with orthogonal, Lorentz, and symplectic symmetries](https://mlsanity.com/paper/2406.01552/) (2024)
- [Tensor Train for Global Optimization Problems in Robotics](https://mlsanity.com/paper/2206.05077/) (2022)
- [Bayesian longitudinal tensor response regression for modeling neuroplasticity](https://mlsanity.com/paper/2309.10065/) (2023)
- [Towards Quantum Tensor Decomposition in Biomedical Applications](https://mlsanity.com/paper/2502.13140/) (2025)
- [Self-Reflective Terrain-Aware Robot Adaptation for Consistent Off-Road Ground Navigation](https://mlsanity.com/paper/2111.06742/) (2021)
- [Modeling Label Correlations for Second-Order Semantic Dependency Parsing with Mean-Field Inference](https://mlsanity.com/paper/2204.03619/) (2022)

## Abstract (arXiv)

Tensors are multidimensional arrays of numerical values and therefore generalize matrices to multiple dimensions. While tensors first emerged in the psychometrics community in the 20th century, they have since then spread to numerous other disciplines, including machine learning. Tensors and their decompositions are especially beneficial in unsupervised learning settings, but are gaining popularity in other sub-disciplines like temporal and multi-relational data analysis, too. The scope of this paper is to give a broad overview of tensors, their decompositions, and how they are used in machine learning. As part of this, we are going to introduce basic tensor concepts, discuss why tensors can be considered more rigid than matrices with respect to the uniqueness of their decomposition, explain the most important factorization algorithms and their properties, provide concrete examples of tensor decomposition applications in machine learning, conduct a case study on tensor-based estimation of mixture models, talk about the current state of research, and provide references to available software libraries.