# Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositions

arXiv: https://arxiv.org/abs/2512.17473 · HTML: https://mlsanity.com/paper/2512.17473/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 19 Dec 2025.

## Builds on (most cited first, 13 of 13)

- [Manopt, a Matlab toolbox for optimization on manifolds](https://mlsanity.com/paper/1308.5200/) (2013)
- [Global Convergence of ADMM in Nonconvex Nonsmooth Optimization](https://mlsanity.com/paper/1511.06324/) (2015)
- [Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems](https://mlsanity.com/paper/1410.1390/) (2014)
- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [Practical sketching algorithms for low-rank matrix approximation](https://mlsanity.com/paper/1609.00048/) (2016)
- [A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization](https://mlsanity.com/paper/1506.04209/) (2015)
- [Positive semidefinite rank](https://mlsanity.com/paper/1407.4095/) (2014)
- [Accelerated Algorithms for Nonlinear Matrix Decomposition with the ReLU function](https://mlsanity.com/paper/2305.08687/) (2023)
- [Bounded Simplex-Structured Matrix Factorization: Algorithms, Identifiability and Applications](https://mlsanity.com/paper/2209.12638/) (2022)
- [The square root rank of the correlation polytope is exponential](https://mlsanity.com/paper/1411.6712/) (2014)
- [Symmetric Matrix Completion with ReLU Sampling](https://mlsanity.com/paper/2406.05822/) (2024)
- [An extrapolated and provably convergent algorithm for nonlinear matrix decomposition with the ReLU function](https://mlsanity.com/paper/2503.23832/) (2025)
- [Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability](https://mlsanity.com/paper/2510.17040/) (2025)

## Built on by (most cited first, 3 of 3)

- [Manifold-based Algorithms for the Hadamard Decomposition](https://mlsanity.com/paper/2605.28980/) (2026)
- [On the Complexity of Low-Rank Matrix Signing and Entrywise Power Matrix Factorization](https://mlsanity.com/paper/2607.04875/) (2026)
- [Subzero matrix completion for sparse data analysis: large-scale learning of latent low-rank structure](https://mlsanity.com/paper/2608.21607/) (2026)

## Abstract (arXiv)

We present an algorithm based on the alternating direction method of multipliers (ADMM) for solving nonlinear matrix decompositions (NMD). Given an input matrix X ∈ ℝm × n and a factorization rank r ll min(m, n), NMD seeks matrices W ∈ ℝm × r and H ∈ ℝr × n such that X ≈ f(WH), where f is an element-wise nonlinear function. We evaluate our method on several representative nonlinear models: the rectified linear unit activation f(x) = max(0, x), suitable for nonnegative sparse data approximation, the component-wise square f(x) = x2, applicable to probabilistic circuit representation, and the MinMax transform f(x) = min(b, max(a, x)), relevant for recommender systems. The proposed framework flexibly supports diverse loss functions, including least squares, ℓ1 norm, and the Kullback-Leibler divergence, and can be readily extended to other nonlinearities and metrics. We illustrate the applicability, efficiency, and adaptability of the approach on real-world datasets, highlighting its potential for a broad range of applications.