# Computing Large-Scale Matrix and Tensor Decomposition with Structured Factors: A Unified Nonconvex Optimization Perspective

arXiv: https://arxiv.org/abs/2006.08183 · HTML: https://mlsanity.com/paper/2006.08183/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 14 of 14)

- [Deep Spectrum Cartography: Completing Radio Map Tensors Using Learned Neural Models](https://mlsanity.com/paper/2105.00177/) (2021)
- [Crowdsourcing via Annotator Co-occurrence Imputation and Provable Symmetric Nonnegative Matrix Factorization](https://mlsanity.com/paper/2106.07193/) (2021)
- [Memory-Efficient Convex Optimization for Self-Dictionary Separable Nonnegative Matrix Factorization: A Frank-Wolfe Approach](https://mlsanity.com/paper/2109.11135/) (2021)
- [Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses](https://mlsanity.com/paper/2104.14562/) (2021)
- [Dual Simplex Volume Maximization for Simplex-Structured Matrix Factorization](https://mlsanity.com/paper/2403.20197/) (2024)
- [Multi-version Tensor Completion for Time-delayed Spatio-temporal Data](https://mlsanity.com/paper/2105.05326/) (2021)
- [Low-Rank Tensors for Multi-Dimensional Markov Models](https://mlsanity.com/paper/2411.02098/) (2024)
- [Personalized Coupled Tensor Decomposition for Multimodal Data Fusion: Uniqueness and Algorithms](https://mlsanity.com/paper/2412.01102/) (2024)
- [Identifiability of Nonnegative Tucker Decompositions -- Part I: Theory](https://mlsanity.com/paper/2505.12713/) (2025)
- [Under-Counted Tensor Completion with Neural Incorporation of Attributes](https://mlsanity.com/paper/2306.03273/) (2023)
- [Structured Low-Rank Tensors for Generalized Linear Models](https://mlsanity.com/paper/2308.02922/) (2023)
- [The Local Landscape of Phase Retrieval Under Limited Samples](https://mlsanity.com/paper/2311.15221/) (2023)
- [Quantum-Inspired Self-Attention in a Large Language Model](https://mlsanity.com/paper/2603.03318/) (2026)
- [Dictionary-based Low-Rank Approximations and the Mixed Sparse Coding problem](https://mlsanity.com/paper/2111.12399/) (2021)

## Abstract (arXiv)

The proposed article aims at offering a comprehensive tutorial for the computational aspects of structured matrix and tensor factorization. Unlike existing tutorials that mainly focus on it algorithmic procedures for a small set of problems, e.g., nonnegativity or sparsity-constrained factorization, we take a it top-down approach: we start with general optimization theory (e.g., inexact and accelerated block coordinate descent, stochastic optimization, and Gauss-Newton methods) that covers a wide range of factorization problems with diverse constraints and regularization terms of engineering interest. Then, we go `under the hood' to showcase specific algorithm design under these introduced principles. We pay a particular attention to recent algorithmic developments in structured tensor and matrix factorization (e.g., random sketching and adaptive step size based stochastic optimization and structure-exploiting second-order algorithms), which are the state of the art---yet much less touched upon in the literature compared to it block coordinate descent (BCD)-based methods. We expect that the article to have an educational values in the field of structured factorization and hope to stimulate more research in this important and exciting direction.