# Tensor Principal Component Analysis in High Dimensional CP Models

arXiv: https://arxiv.org/abs/2108.04428 · HTML: https://mlsanity.com/paper/2108.04428/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 68 references that are in the map.
- First posted on arXiv 10 Aug 2021.

## Builds on (most cited first, 26 of 26)

- [A statistical model for tensor PCA](https://mlsanity.com/paper/1411.1076/) (2014)
- [Tensor SVD: Statistical and Computational Limits](https://mlsanity.com/paper/1703.02724/) (2017)
- [Reducibility and Statistical-Computational Gaps from Secret Leakage](https://mlsanity.com/paper/2005.08099/) (2020)
- [An Optimal Statistical and Computational Framework for Generalized Tensor Estimation](https://mlsanity.com/paper/2002.11255/) (2020)
- [Nonconvex Low-Rank Tensor Completion from Noisy Data](https://mlsanity.com/paper/1911.04436/) (2019)
- [Multilinear tensor regression for longitudinal relational data](https://mlsanity.com/paper/1412.0048/) (2014)
- [Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements](https://mlsanity.com/paper/2104.14526/) (2021)
- [On Polynomial Time Methods for Exact Low Rank Tensor Completion](https://mlsanity.com/paper/1702.06980/) (2017)
- [Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updates](https://mlsanity.com/paper/1402.5180/) (2014)
- [Factor Models for High-Dimensional Tensor Time Series](https://mlsanity.com/paper/1905.07530/) (2019)
- [Provable Sparse Tensor Decomposition](https://mlsanity.com/paper/1502.01425/) (2015)
- [Multilayer tensor factorization with applications to recommender systems](https://mlsanity.com/paper/1711.01598/) (2017)
- [Tensor Factorization via Matrix Factorization](https://mlsanity.com/paper/1501.07320/) (2015)
- [Constrained Factor Models for High-Dimensional Matrix-Variate Time Series](https://mlsanity.com/paper/1710.06075/) (2017)
- [Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergence](https://mlsanity.com/paper/2104.12031/) (2021)
- [STORE: Sparse Tensor Response Regression and Neuroimaging Analysis](https://mlsanity.com/paper/1609.04523/) (2016)
- [Sparse and Low-rank Tensor Estimation via Cubic Sketchings](https://mlsanity.com/paper/1801.09326/) (2018)
- [Statistical Inference for High-Dimensional Matrix-Variate Factor Model](https://mlsanity.com/paper/2001.01890/) (2020)
- [Guaranteed Functional Tensor Singular Value Decomposition](https://mlsanity.com/paper/2108.04201/) (2021)
- [A Tensor Approach to Learning Mixed Membership Community Models](https://mlsanity.com/paper/1302.2684/) (2013)
- [Tensor Factor Model Estimation by Iterative Projection](https://mlsanity.com/paper/2006.02611/) (2020)
- [Rank Determination in Tensor Factor Model](https://mlsanity.com/paper/2011.07131/) (2020)
- [Optimal Sparse Singular Value Decomposition for High-dimensional High-order Data](https://mlsanity.com/paper/1809.01796/) (2018)
- [Online Tensor Methods for Learning Latent Variable Models](https://mlsanity.com/paper/1309.0787/) (2013)
- [Projected Estimation for Large-dimensional Matrix Factor Models](https://mlsanity.com/paper/2003.10285/) (2020)
- [Tensor Decompositions via Two-Mode Higher-Order SVD (HOSVD)](https://mlsanity.com/paper/1612.03839/) (2016)

## Built on by (most cited first, 7 of 7)

- [Deflated HeteroPCA: Overcoming the curse of ill-conditioning in heteroskedastic PCA](https://mlsanity.com/paper/2303.06198/) (2023)
- [Factor Augmented Tensor-on-Tensor Neural Networks](https://mlsanity.com/paper/2405.19610/) (2024)
- [Modelling matrix time series via a tensor CP-decomposition](https://mlsanity.com/paper/2112.15423/) (2021)
- [Covariate-Adjusted Deep Causal Learning for Heterogeneous Panel Data Models](https://mlsanity.com/paper/2505.20536/) (2025)
- [TEAFormers: TEnsor-Augmented Transformers for Multi-Dimensional Time Series Forecasting](https://mlsanity.com/paper/2410.20439/) (2024)
- [Supervised Dynamic Dimension Reduction with Deep Neural Network](https://mlsanity.com/paper/2508.03546/) (2025)
- [Guaranteed Noisy CP Tensor Recovery via Riemannian Optimization on the Segre Manifold](https://mlsanity.com/paper/2510.00569/) (2025)

## Abstract (arXiv)

The CP decomposition for high dimensional non-orthogonal spiked tensors is an important problem with broad applications across many disciplines. However, previous works with theoretical guarantee typically assume restrictive incoherence conditions on the basis vectors for the CP components. In this paper, we propose new computationally efficient composite PCA and concurrent orthogonalization algorithms for tensor CP decomposition with theoretical guarantees under mild incoherence conditions. The composite PCA applies the principal component or singular value decompositions twice, first to a matrix unfolding of the tensor data to obtain singular vectors and then to the matrix folding of the singular vectors obtained in the first step. It can be used as an initialization for any iterative optimization schemes for the tensor CP decomposition. The concurrent orthogonalization algorithm iteratively estimates the basis vector in each mode of the tensor by simultaneously applying projections to the orthogonal complements of the spaces generated by other CP components in other modes. It is designed to improve the alternating least squares estimator and other forms of the high order orthogonal iteration for tensors with low or moderately high CP ranks, and it is guaranteed to converge rapidly when the error of any given initial estimator is bounded by a small constant. Our theoretical investigation provides estimation accuracy and convergence rates for the two proposed algorithms. Both proposed algorithms are applicable to deterministic tensor, its noisy version, and the order-2K covariance tensor of order-K tensor data in a factor model with uncorrelated factors. Our implementations on synthetic data demonstrate significant practical superiority of our approach over existing methods.