# DeepTensor: Low-Rank Tensor Decomposition with Deep Network Priors

arXiv: https://arxiv.org/abs/2204.03145 · HTML: https://mlsanity.com/paper/2204.03145/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 68 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Snapshot Ensembles: Train 1, get M for free](https://mlsanity.com/paper/1704.00109/) (2017)
- [Implicit Bias of Gradient Descent on Linear Convolutional Networks](https://mlsanity.com/paper/1806.00468/) (2018)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination](https://mlsanity.com/paper/1401.6497/) (2014)
- [Hyperspectral Image Restoration via Total Variation Regularized Low-rank Tensor Decomposition](https://mlsanity.com/paper/1707.02477/) (2017)
- [Robust Low-rank Tensor Recovery: Models and Algorithms](https://mlsanity.com/paper/1311.6182/) (2013)
- [Bayesian Robust Tensor Factorization for Incomplete Multiway Data](https://mlsanity.com/paper/1410.2386/) (2014)
- [Deep Plug-and-play Prior for Low-rank Tensor Completion](https://mlsanity.com/paper/1905.04449/) (2019)
- [Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization](https://mlsanity.com/paper/1912.02314/) (2019)
- [Robust Kronecker Component Analysis](https://mlsanity.com/paper/1801.06432/) (2018)
- [Compressive Spectral Image Reconstruction using Deep Prior and Low-Rank Tensor Representation](https://mlsanity.com/paper/2101.07424/) (2021)
- [KRISM --- Krylov Subspace-based Optical Computing of Hyperspectral Images](https://mlsanity.com/paper/1801.09343/) (2018)
- [Deep Low-rank Prior in Dynamic MR Imaging](https://mlsanity.com/paper/2006.12090/) (2020)
- [Low Rank Forecasting](https://mlsanity.com/paper/2101.12414/) (2021)

## Built on by (most cited first, 15 of 15)

- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models](https://mlsanity.com/paper/2402.02244/) (2024)
- [F-INR: Functional Tensor Decomposition for Implicit Neural Representations](https://mlsanity.com/paper/2503.21507/) (2025)
- [Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization](https://mlsanity.com/paper/2506.22134/) (2025)
- [Reparameterized Tensor Ring Functional Decomposition for Multi-Dimensional Data Recovery](https://mlsanity.com/paper/2603.01034/) (2026)
- [Enabling Tensor Decomposition for Time-Series Classification via A Simple Pseudo-Laplacian Contrast](https://mlsanity.com/paper/2409.15200/) (2024)
- [Academic Network Representation via Prediction-Sampling Incorporated Tensor Factorization](https://mlsanity.com/paper/2504.08323/) (2025)
- [Score-Based Model for Low-Rank Tensor Recovery](https://mlsanity.com/paper/2506.22295/) (2025)
- [MGAA: Multi-Granular Adaptive Allocation fof Low-Rank Compression of LLMs](https://mlsanity.com/paper/2507.03294/) (2025)
- [A Single Image Is All You Need: Zero-Shot Anomaly Localization Without Training Data](https://mlsanity.com/paper/2509.18354/) (2025)
- [StoTAM: Stochastic Alternating Minimization for Tucker-Structured Tensor Sensing](https://mlsanity.com/paper/2601.13522/) (2026)
- [CogGen: Cognitive-Load-Inspired Fully Unsupervised Deep Generative Modeling for Compressively Sampled MRI Reconstruction](https://mlsanity.com/paper/2603.04438/) (2026)
- [Efficient Generative Modeling with Unitary Matrix Product States Using Riemannian Optimization](https://mlsanity.com/paper/2603.12026/) (2026)
- [Bayesian Tensor Decomposition with Diffusion Model Prior](https://mlsanity.com/paper/2606.03212/) (2026)
- [Pre-Trained Low-Rank Tensor Decomposition for Multi-Dimensional Image Recovery](https://mlsanity.com/paper/2609.12843/) (2026)

## Abstract (arXiv)

DeepTensor is a computationally efficient framework for low-rank decomposition of matrices and tensors using deep generative networks. We decompose a tensor as the product of low-rank tensor factors (e.g., a matrix as the outer product of two vectors), where each low-rank tensor is generated by a deep network (DN) that is trained in a self-supervised manner to minimize the mean-squared approximation error. Our key observation is that the implicit regularization inherent in DNs enables them to capture nonlinear signal structures (e.g., manifolds) that are out of the reach of classical linear methods like the singular value decomposition (SVD) and principal component analysis (PCA). Furthermore, in contrast to the SVD and PCA, whose performance deteriorates when the tensor's entries deviate from additive white Gaussian noise, we demonstrate that the performance of DeepTensor is robust to a wide range of distributions. We validate that DeepTensor is a robust and computationally efficient drop-in replacement for the SVD, PCA, nonnegative matrix factorization (NMF), and similar decompositions by exploring a range of real-world applications, including hyperspectral image denoising, 3D MRI tomography, and image classification. In particular, DeepTensor offers a 6dB signal-to-noise ratio improvement over standard denoising methods for signals corrupted by Poisson noise and learns to decompose 3D tensors 60 times faster than a single DN equipped with 3D convolutions.