mlsanityData to 7 Oct 2026

On subdifferential chain rule of matrix factorization and beyond

arXiv 2410.05022 ↗First posted 7 Oct 2024math.OC stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we study equality-type Clarke subdifferential chain rules of matrix factorization and factorization machine. Specifically, we show for these problems that provided the latent dimension is larger than some multiple of the problem size (i.e., slightly overparameterized) and the loss function is locally Lipschitz, the subdifferential chain rules hold everywhere. In addition, we examine the tightness of the analysis through some interesting constructions and make some important observations from the perspective of optimization; e.g., we show that for all this type of problems, computing a stationary point is trivial. Some tensor generalizations and neural extensions are also discussed, albeit they remain mostly open.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 55 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

25 of 55 references.

References · 25

  1. 2019Graph Neural Networks for Social Recommendationcited by 711
  2. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  3. 2017Neural Factorization Machines for Sparse Predictive Analyticscited by 183
  4. 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 172
  5. 2017Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networkscited by 126
  6. 2014Matrix Completion and Low-Rank SVD via Fast Alternating Least Squarescited by 104
  7. 2017Learning ReLUs via Gradient Descentcited by 54
  8. 2018NAIS: Neural Attentive Item Similarity Model for Recommendationcited by 52
  9. 2016Higher-Order Factorization Machinescited by 45
  10. 2021Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorizationcited by 44
  11. 2021On the Finite-Time Complexity and Practical Computation of Approximate Stationarity Concepts of Lipschitz Functionscited by 30
  12. 2019Relational Collaborative Filtering:Modeling Multiple Item Relations for Recommendationcited by 23
13 more
  1. 2022The effect of smooth parametrizations on nonconvex optimization landscapescited by 20
  2. 2023The Law of Parsimony in Gradient Descent for Learning Deep Linear Networkscited by 19
  3. 2020Understanding Notions of Stationarity in Non-Smooth Optimizationcited by 18
  4. 2018Dropping Symmetry for Fast Symmetric Nonnegative Matrix Factorizationcited by 17
  5. 2017Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms, and Applicationscited by 16
  6. 2021Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorizationcited by 15
  7. 2022No Dimension-Free Deterministic Algorithm Computes Approximate Stationarities of Lipschitzianscited by 15
  8. 2016Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithmscited by 11
  9. 2017Multi-output Polynomial Networks and Factorization Machinescited by 8
  10. 2018Efficiently testing local optimality and escaping saddles for ReLU networkscited by 4
  11. 2024Symmetric Matrix Completion with ReLU Samplingcited by 4
  12. 2020Factorization Machines with Regularization for Sparse Feature Interactionscited by 3
  13. 2023Testing Stationarity Concepts for ReLU Networks: Hardness, Regularity, and Robust Algorithmscited by 2

Built on by

Not cited yet.