# Fast and Effective Computation of Generalized Symmetric Matrix Factorization

arXiv: https://arxiv.org/abs/2603.19147 · HTML: https://mlsanity.com/paper/2603.19147/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 19 Mar 2026.

## Builds on (most cited first, 16 of 16)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Speeding up Convolutional Neural Networks with Low Rank Expansions](https://mlsanity.com/paper/1405.3866/) (2014)
- [QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2309.14717/) (2023)
- [Generalized Low Rank Models](https://mlsanity.com/paper/1410.0342/) (2014)
- [Calculus of the exponent of Kurdyka-Lojasiewicz inequality and its applications to linear convergence of first-order methods](https://mlsanity.com/paper/1602.02915/) (2016)
- [A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems](https://mlsanity.com/paper/1303.4434/) (2013)
- [Low-Rank Modeling and Its Applications in Image Analysis](https://mlsanity.com/paper/1401.3409/) (2014)
- [An Inexact Projected Gradient Method with Rounding and Lifting by Nonlinear Programming for Solving Rank-One Semidefinite Relaxation of Polynomial Optimization](https://mlsanity.com/paper/2105.14033/) (2021)
- [A Provable Splitting Approach for Symmetric Nonnegative Matrix Factorization](https://mlsanity.com/paper/2301.10499/) (2023)
- [Accelerating nuclear-norm regularized low-rank matrix optimization through Burer-Monteiro decomposition](https://mlsanity.com/paper/2204.14067/) (2022)
- [Scalable Algorithms for Tractable Schatten Quasi-Norm Minimization](https://mlsanity.com/paper/1606.01245/) (2016)
- [Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems](https://mlsanity.com/paper/1711.06831/) (2017)
- [Tractable and Scalable Schatten Quasi-Norm Approximations for Rank Minimization](https://mlsanity.com/paper/1803.00420/) (2018)
- [A nonmonotone extrapolated proximal gradient-subgradient algorithm beyond global Lipschitz gradient continuity](https://mlsanity.com/paper/2511.22011/) (2025)
- [A Non-monotone Alternating Updating Method for A Class of Matrix Factorization Problems](https://mlsanity.com/paper/1705.06499/) (2017)
- [Convergence of ZH-type nonmonotone descent method for Kurdyka-Lojasiewicz optimization problems](https://mlsanity.com/paper/2406.05740/) (2024)

## Abstract (arXiv)

In this paper, we study a nonconvex, nonsmooth, and non-Lipschitz generalized symmetric matrix factorization model that unifies a broad class of matrix factorization formulations arising in machine learning, image science, engineering, and related areas. We first establish two exactness properties. On the modeling side, we prove an exact penalty property showing that, under suitable conditions, the symmetry-inducing quadratic penalty enforces symmetry whenever the penalty parameter is sufficiently large but finite, thereby exactly recovering the associated symmetric formulation. On the algorithmic side, we introduce an auxiliary-variable splitting formulation and establish an exact relaxation relationship that rigorously links stationary points of the original objective function to those of a relaxed potential function. Building on these exactness properties, we propose an average-type nonmonotone alternating updating method (A-NAUM) based on the relaxed potential function. At each iteration, A-NAUM alternately updates the two factor blocks by (approximately) minimizing the potential function, while the auxiliary block is updated in closed form. To ensure the convergence and enhance practical performance, we further incorporate an average-type nonmonotone line search and show that it is well-defined under mild conditions. Moreover, based on the Kurdyka-Lojasiewicz property and its associated exponent, we establish global convergence of the entire sequence to a stationary point and derive convergence rate results. Finally, numerical experiments on real datasets demonstrate the efficiency of A-NAUM.