# Joint Majorization-Minimization for Nonnegative Matrix Factorization with the β-divergence

arXiv: https://arxiv.org/abs/2106.15214 · HTML: https://mlsanity.com/paper/2106.15214/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 38 references that are in the map.
- First posted on arXiv 29 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [Nonnegative Matrix Factorization for Signal and Data Analytics: Identifiability, Algorithms, and Applications](https://mlsanity.com/paper/1803.01257/) (2018)
- [First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems](https://mlsanity.com/paper/1706.06461/) (2017)
- [Nonlinear hyperspectral unmixing with robust nonnegative matrix factorization](https://mlsanity.com/paper/1401.5649/) (2014)
- [Ordinal Non-negative Matrix Factorization for Recommendation](https://mlsanity.com/paper/2006.01034/) (2020)
- [Stochastic Subsampling for Factorizing Huge Matrices](https://mlsanity.com/paper/1701.05363/) (2017)
- [Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms](https://mlsanity.com/paper/1905.09050/) (2019)
- [Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses](https://mlsanity.com/paper/2104.14562/) (2021)
- [A Unified Convergence Analysis of the Multiplicative Update Algorithm for Regularized Nonnegative Matrix Factorization](https://mlsanity.com/paper/1609.00951/) (2016)
- [Majorization-Minimization-Based Levenberg--Marquardt Method for Constrained Nonlinear Least Squares](https://mlsanity.com/paper/2004.08259/) (2020)
- [Convergence to Second-Order Stationarity for Non-negative Matrix Factorization: Provably and Concurrently](https://mlsanity.com/paper/2002.11323/) (2020)

## Built on by (most cited first, 7 of 7)

- [A Second-Order Majorant Algorithm for Nonnegative Matrix Factorization](https://mlsanity.com/paper/2303.17992/) (2023)
- [Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models](https://mlsanity.com/paper/2403.18517/) (2024)
- [Nonnegative Tucker Decomposition with Beta-divergence for Music Structure Analysis of Audio Signals](https://mlsanity.com/paper/2110.14434/) (2021)
- [Block Majorization Minimization with Extrapolation and Application to β-NMF](https://mlsanity.com/paper/2401.06646/) (2024)
- [Dictionary-based Low-Rank Approximations and the Mixed Sparse Coding problem](https://mlsanity.com/paper/2111.12399/) (2021)
- [On the Connection Between Non-negative Matrix Factorization and Latent Dirichlet Allocation](https://mlsanity.com/paper/2405.20542/) (2024)
- [An Efficient Newton Algorithm for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence](https://mlsanity.com/paper/2607.13919/) (2026)

## Abstract (arXiv)

This article proposes new multiplicative updates for nonnegative matrix factorization (NMF) with the β-divergence objective function. Our new updates are derived from a joint majorization-minimization (MM) scheme, in which an auxiliary function (a tight upper bound of the objective function) is built for the two factors jointly and minimized at each iteration. This is in contrast with the classic approach in which a majorizer is derived for each factor separately. Like that classic approach, our joint MM algorithm also results in multiplicative updates that are simple to implement. They however yield a significant drop of computation time (for equally good solutions), in particular for some β-divergences of important applicative interest, such as the squared Euclidean distance and the Kullback-Leibler or Itakura-Saito divergences. We report experimental results using diverse datasets: face images, an audio spectrogram, hyperspectral data and song play counts. Depending on the value of β and on the dataset, our joint MM approach can yield CPU time reductions from about 13% to 78% in comparison to the classic alternating scheme.