# Weighted Low-Rank Matrix Approximation: Acceleration and Applications

arXiv: https://arxiv.org/abs/2109.11057 · HTML: https://mlsanity.com/paper/2109.11057/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 31 references that are in the map.
- First posted on arXiv 22 Sep 2021.

## Builds on (most cited first, 6 of 6)

- [Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares](https://mlsanity.com/paper/1410.2596/) (2014)
- [Generalized Low Rank Models](https://mlsanity.com/paper/1410.0342/) (2014)
- [Low-Rank Matrix Completion: A Contemporary Survey](https://mlsanity.com/paper/1907.11705/) (2019)
- [Globally Convergent Type-I Anderson Acceleration for Non-Smooth Fixed-Point Iterations](https://mlsanity.com/paper/1808.03971/) (2018)
- [Recovery guarantee of weighted low-rank approximation via alternating minimization](https://mlsanity.com/paper/1602.02262/) (2016)
- [Generalized Matrix Factorization: efficient algorithms for fitting generalized linear latent variable models to large data arrays](https://mlsanity.com/paper/2010.02469/) (2020)

## Built on by (most cited first, 2 of 2)

- [LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning](https://mlsanity.com/paper/2311.12023/) (2023)
- [When Can We Solve the Weighted Low Rank Approximation Problem in Truly Subquadratic Time?](https://mlsanity.com/paper/2502.16912/) (2025)

## Abstract (arXiv)

Weighted low-rank matrix approximation (WLRMA) generalizes classical low-rank approximation and matrix completion by allowing arbitrary elementwise weights. Such formulations arise naturally in a broad class of statistical models, including generalized linear low-rank models, where WLRMA serves as the computational primitive for parameter estimation. Despite its broad applicability, efficient optimization methods for general WLRMA remain relatively underdeveloped. In this paper, we formulate both the rank-constrained and nuclear-norm WLRMA problems within a unified first-order optimization framework by showing that the corresponding iterative algorithms are projected and proximal gradient methods. Building on this perspective, we develop accelerated algorithms based on Nesterov momentum and Anderson acceleration, together with a regularized Anderson scheme that improves numerical stability for non-convex problems. We further propose scalable implementations for large sparse data matrices and introduce a practical effective-rank criterion that provides a meaningful correspondence between rank-constrained and nuclear-norm solutions. We further show that fitting generalized linear low-rank models can be reduced to a sequence of weighted low-rank matrix approximation problems, allowing the proposed algorithms to be used as computational building blocks for their estimation. Simulation studies demonstrate substantial computational gains achieved by the proposed accelerated algorithms. Applications to the MovieLens dataset further illustrate the proposed framework for matrix completion, heteroscedastic Gaussian low-rank modeling, and logistic low-rank modeling.