mlsanityData to 7 Oct 2026

Implicit Regularization with Polynomial Growth in Deep Tensor Factorization

arXiv 2207.08942 ↗First posted 18 Jul 2022cs.LG cs.AI cs.NEcited by 3

Not on the map. Why?

Abstract (arXiv)

We study the implicit regularization effects of deep learning in tensor factorization. While implicit regularization in deep matrix and 'shallow' tensor factorization via linear and certain type of non-linear neural networks promotes low-rank solutions with at most quadratic growth, we show that its effect in deep tensor factorization grows polynomially with the depth of the network. This provides a remarkably faithful description of the observed experimental behaviour. Using numerical experiments, we demonstrate the benefits of this implicit regularization in yielding a more accurate estimation and better convergence properties.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 18 Jul 2022.

Builds on

16 of 22 references.

References · 16

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  2. 2019Implicit Regularization in Deep Matrix Factorizationcited by 402
  3. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  4. 2017Implicit Regularization in Matrix Factorizationcited by 318
  5. 2018On the Optimization of Deep Networks: Implicit Acceleration by Overparameterizationcited by 280
  6. 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 172
  7. 2020Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learningcited by 148
  8. 2020Implicit Regularization in Deep Learning May Not Be Explainable by Normscited by 118
  9. 2017Tensor Completion Algorithms in Big Data Analyticscited by 48
  10. 2021Implicit Regularization in Tensor Factorizationcited by 43
  11. 2020Implicit Rank-Minimizing Autoencodercited by 35
  12. 2020On Implicit Regularization in β-VAEscited by 34
4 more
  1. 2021The Benefits of Implicit Regularization from SGD in Least Squares Problemscited by 32
  2. 2022Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networkscited by 27
  3. 2021Understanding Deflation Process in Over-parametrized Tensor Decompositioncited by 20
  4. 2021Implicit Regularization in Deep Tensor Factorizationcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 2.

3 papers.

  1. 2022Incremental Learning in Diagonal Linear Networkscited by 27
  2. 2025Continuous Representation Methods, Theories, and Applications: An Overview and Perspectivescited by 4
  3. 2025Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Modelscited by 0