# Extension of PCA to Higher Order Data Structures: An Introduction to Tensors, Tensor Decompositions, and Tensor PCA

arXiv: https://arxiv.org/abs/1803.00704 · HTML: https://mlsanity.com/paper/1803.00704/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2018.

## Built on by (most cited first, 14 of 14)

- [Tensor Networks Meet Neural Networks: A Survey and Future Perspectives](https://mlsanity.com/paper/2302.09019/) (2023)
- [Mode-wise Tensor Decompositions: Multi-dimensional Generalizations of CUR Decompositions](https://mlsanity.com/paper/2103.11037/) (2021)
- [When Random Tensors meet Random Matrices](https://mlsanity.com/paper/2112.12348/) (2021)
- [Efficient Tensor Robust PCA under Hybrid Model of Tucker and Tensor Train](https://mlsanity.com/paper/2112.10771/) (2021)
- [Optimizing Orthogonalized Tensor Deflation via Random Tensor Theory](https://mlsanity.com/paper/2302.05798/) (2023)
- [Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Models](https://mlsanity.com/paper/2405.06626/) (2024)
- [Tensor-variate Mixture of Experts for Proportional Myographic Control of a Robotic Hand](https://mlsanity.com/paper/1902.11104/) (2019)
- [Multivariate Analysis for Multiple Network Data via Semi-Symmetric Tensor PCA](https://mlsanity.com/paper/2202.04719/) (2022)
- [Reduced-order autoregressive dynamics of a complex financial system: a PCA-based approach](https://mlsanity.com/paper/2212.12044/) (2022)
- [Latent Matrices for Tensor Network Decomposition and to Tensor Completion](https://mlsanity.com/paper/2210.03392/) (2022)
- [Application of Tensor Neural Networks to Pricing Bermudan Swaptions](https://mlsanity.com/paper/2304.09750/) (2023)
- [Tensor PCA from basis in tensor space](https://mlsanity.com/paper/2305.02803/) (2023)
- [Decomposition of linear tensor transformations](https://mlsanity.com/paper/2309.07819/) (2023)
- [Improving LSH via Tensorized Random Projection](https://mlsanity.com/paper/2402.07189/) (2024)

## Abstract (arXiv)

The widespread use of multisensor technology and the emergence of big data sets have brought the necessity to develop more versatile tools to represent higher-order data with multiple aspects and high dimensionality. Data in the form of multidimensional arrays, also referred to as tensors, arises in a variety of applications including chemometrics, hyperspectral imaging, high resolution videos, neuroimaging, biometrics, and social network analysis. Early multiway data analysis approaches reformatted such tensor data as large vectors or matrices and then resorted to dimensionality reduction methods developed for classical two-way analysis such as PCA. However, one cannot discover hidden components within multiway data using conventional PCA. To this end, tensor decomposition methods which are flexible in the choice of the constraints and that extract more general latent components have been proposed. In this paper, we review the major tensor decomposition methods with a focus on problems targeted by classical PCA. In particular, we present tensor methods that aim to solve three important challenges typically addressed by PCA: dimensionality reduction, i.e. low-rank tensor approximation, supervised learning, i.e. learning linear subspaces for feature extraction, and robust low-rank tensor recovery. We also provide experimental results to compare different tensor models for both dimensionality reduction and supervised learning applications.