# Stochastic Gradients for Large-Scale Tensor Decomposition

arXiv: https://arxiv.org/abs/1906.01687 · HTML: https://mlsanity.com/paper/1906.01687/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 16 of 16)

- [An AO-ADMM approach to constraining PARAFAC2 on all modes](https://mlsanity.com/paper/2110.01278/) (2021)
- [Convergence of Alternating Gradient Descent for Matrix Factorization](https://mlsanity.com/paper/2305.06927/) (2023)
- [ATD: Augmenting CP Tensor Decomposition by Self Supervision](https://mlsanity.com/paper/2106.07900/) (2021)
- [F-INR: Functional Tensor Decomposition for Implicit Neural Representations](https://mlsanity.com/paper/2503.21507/) (2025)
- [Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses](https://mlsanity.com/paper/2104.14562/) (2021)
- [Zero-Truncated Poisson Regression for Sparse Multiway Count Data Corrupted by False Zeros](https://mlsanity.com/paper/2201.10014/) (2022)
- [Stochastic regularized majorization-minimization with weakly convex and multi-convex surrogates](https://mlsanity.com/paper/2201.01652/) (2022)
- [Comparing Two Proxy Methods for Causal Identification](https://mlsanity.com/paper/2512.00175/) (2025)
- [Near-Universal Multiplicative Updates for Nonnegative Einsum Factorization](https://mlsanity.com/paper/2602.02759/) (2026)
- [Streaming Generalized Canonical Polyadic Tensor Decompositions](https://mlsanity.com/paper/2110.14514/) (2021)
- [Tensor Decomposition with Unaligned Observations](https://mlsanity.com/paper/2410.14046/) (2024)
- [Communication Efficient Generalized Tensor Factorization for Decentralized Healthcare Networks](https://mlsanity.com/paper/2109.01718/) (2021)
- [Reconfigurable Low-latency Memory System for Sparse Matricized Tensor Times Khatri-Rao Product on FPGA](https://mlsanity.com/paper/2109.08874/) (2021)
- [Dictionary-based Low-Rank Approximations and the Mixed Sparse Coding problem](https://mlsanity.com/paper/2111.12399/) (2021)
- [Generalized Canonical Polyadic Tensor Decompositions with General Symmetry](https://mlsanity.com/paper/2601.05335/) (2026)
- [StoTAM: Stochastic Alternating Minimization for Tucker-Structured Tensor Sensing](https://mlsanity.com/paper/2601.13522/) (2026)

## Abstract (arXiv)

Tensor decomposition is a well-known tool for multiway data analysis. This work proposes using stochastic gradients for efficient generalized canonical polyadic (GCP) tensor decomposition of large-scale tensors. GCP tensor decomposition is a recently proposed version of tensor decomposition that allows for a variety of loss functions such as Bernoulli loss for binary data or Huber loss for robust estimation. The stochastic gradient is formed from randomly sampled elements of the tensor and is efficient because it can be computed using the sparse matricized-tensor-times-Khatri-Rao product (MTTKRP) tensor kernel. For dense tensors, we simply use uniform sampling. For sparse tensors, we propose two types of stratified sampling that give precedence to sampling nonzeros. Numerical results demonstrate the advantages of the proposed approach and its scalability to large-scale problems.