# Nonconvex Optimization via MM Algorithms: Convergence Theory

arXiv: https://arxiv.org/abs/2106.02805 · HTML: https://mlsanity.com/paper/2106.02805/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2021.

## Built on by (most cited first, 5 of 5)

- [Policy Optimization for Continuous Reinforcement Learning](https://mlsanity.com/paper/2305.18901/) (2023)
- [Accurate, Interpretable, and Fast Animation: An Iterative, Sparse, and Nonconvex Approach](https://mlsanity.com/paper/2109.08356/) (2021)
- [Learning Massive-scale Partial Correlation Networks in Clinical Multi-omics Studies with HP-ACCORD](https://mlsanity.com/paper/2412.11554/) (2024)
- [Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Experts](https://mlsanity.com/paper/2601.19811/) (2026)
- [Fast Model Selection and Stable Optimization for Softmax-Gated Multinomial-Logistic Mixture of Experts Models](https://mlsanity.com/paper/2602.07997/) (2026)

## Abstract (arXiv)

The majorization-minimization (MM) principle is an extremely general framework for deriving optimization algorithms. It includes the expectation-maximization (EM) algorithm, proximal gradient algorithm, concave-convex procedure, quadratic lower bound algorithm, and proximal distance algorithm as special cases. Besides numerous applications in statistics, optimization, and imaging, the MM principle finds wide applications in large scale machine learning problems such as matrix completion, discriminant analysis, and nonnegative matrix factorizations. When applied to nonconvex optimization problems, MM algorithms enjoy the advantages of convexifying the objective function, separating variables, numerical stability, and ease of implementation. However, compared to the large body of literature on other optimization algorithms, the convergence analysis of MM algorithms is scattered and problem specific. This survey presents a unified treatment of the convergence of MM algorithms. With modern applications in mind, the results encompass non-smooth objective functions and non-asymptotic analysis.