mlsanityData to 7 Oct 2026

Computational and Statistical Guarantees for Tensor-on-Tensor Regression with Tensor Train Decomposition

arXiv 2406.06002 ↗First posted 10 Jun 2024cs.LG eess.SP math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, a tensor-on-tensor (ToT) regression model has been proposed to generalize tensor recovery, encompassing scenarios like scalar-on-tensor regression and tensor-on-vector regression. However, the exponential growth in tensor complexity poses challenges for storage and computation in ToT regression. To overcome this hurdle, tensor decompositions have been introduced, with the tensor train (TT)-based ToT model proving efficient in practice due to reduced memory requirements, enhanced computational efficiency, and decreased sampling complexity. Despite these practical benefits, a disparity exists between theoretical analysis and real-world performance. In this paper, we delve into the theoretical and algorithmic aspects of the TT-based ToT regression model. Assuming the regression operator satisfies the restricted isometry property (RIP), we conduct an error analysis for the solution to a constrained least-squares optimization problem. This analysis includes upper error bound and minimax lower bound, revealing that such error bounds polynomially depend on the order N+M. To efficiently find solutions meeting such error bounds, we propose two optimization algorithms: the iterative hard thresholding (IHT) algorithm (employing gradient descent with TT-singular value decomposition (TT-SVD)) and the factorization approach using the Riemannian gradient descent (RGD) algorithm. When RIP is satisfied, spectral initialization facilitates proper initialization, and we establish the linear convergence rate of both IHT and RGD.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 29 of its 70 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

29 of 70 references.

References · 29

  1. 2016Tensor Decomposition for Signal Processing and Machine Learningcited by 223
  2. 2018Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overviewcited by 159
  3. 2014Phase Retrieval via Wirtinger Flow: Theory and Algorithmscited by 123
  4. 2014Guaranteed Matrix Completion via Non-convex Factorizationcited by 81
  5. 2020Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descentcited by 70
  6. 2017Tensor SVD: Statistical and Computational Limitscited by 52
  7. 2017Tensor Regression Networkscited by 44
  8. 2020An Optimal Statistical and Computational Framework for Generalized Tensor Estimationcited by 39
  9. 2017Global Optimality in Low-rank Matrix Optimizationcited by 35
  10. 2014Multilinear tensor regression for longitudinal relational datacited by 28
  11. 2021Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurementscited by 28
  12. 2017On Polynomial Time Methods for Exact Low Rank Tensor Completioncited by 27
17 more
  1. 2020Low-Rank Matrix Recovery with Scaled Subgradient Methods: Fast and Robust Convergence Without the Condition Numbercited by 27
  2. 2017Phase Transitions of Spectral Initialization for High-Dimensional Nonconvex Estimationcited by 26
  3. 2017The Global Optimization Geometry of Low-Rank Matrix Optimizationcited by 24
  4. 2017Tensor-on-tensor regressioncited by 22
  5. 2015Parsimonious Tensor Response Regressioncited by 20
  6. 2021Provable Tensor-Train Format Tensor Completion by Riemannian Optimizationcited by 17
  7. 2017Multilayer tensor factorization with applications to recommender systemscited by 16
  8. 2022Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplaycited by 15
  9. 2018Fast and Accurate Tensor Completion with Total Variation Regularized Tensor Trainscited by 14
  10. 2021Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergencecited by 13
  11. 2016STORE: Sparse Tensor Response Regression and Neuroimaging Analysiscited by 11
  12. 2023Robust Tensor CUR Decompositions: Rapid Low-Tucker-Rank Tensor Recovery with Sparse Corruptioncited by 11
  13. 2020Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variancecited by 10
  14. 2019Iterative Hard Thresholding for Low CP-rank Tensor Modelscited by 9
  15. 2023Quantum State Tomography for Matrix Product Density Operatorscited by 9
  16. 2024Guaranteed Nonconvex Factorization Approach for Tensor Train Recoverycited by 8
  17. 2021Tensor train completion: local recovery guarantees via Riemannian optimizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025A Scalable Factorization Approach for High-Order Structured Tensor Recoverycited by 4
  2. 2024Robust Low-rank Tensor Train Recoverycited by 2
  3. 2025Efficient Low-Tubal-Rank Tensor Estimation via Alternating Preconditioned Gradient Descentcited by 1
  4. 2026Structured Adaptive Tensor Prediction for Streaming Datacited by 0