# Manifold-based Algorithms for the Hadamard Decomposition

arXiv: https://arxiv.org/abs/2605.28980 · HTML: https://mlsanity.com/paper/2605.28980/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 16 of 16)

- [FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2108.06098/) (2021)
- [Manopt, a Matlab toolbox for optimization on manifolds](https://mlsanity.com/paper/1308.5200/) (2013)
- [Generalized Low Rank Models](https://mlsanity.com/paper/1410.0342/) (2014)
- [Practical sketching algorithms for low-rank matrix approximation](https://mlsanity.com/paper/1609.00048/) (2016)
- [Block Coordinate Descent on Smooth Manifolds: Convergence Theory and Twenty-One Examples](https://mlsanity.com/paper/2305.14744/) (2023)
- [Positive semidefinite rank](https://mlsanity.com/paper/1407.4095/) (2014)
- [Accelerating Nonnegative Matrix Factorization Algorithms using Extrapolation](https://mlsanity.com/paper/1805.06604/) (2018)
- [Accelerated Algorithms for Nonlinear Matrix Decomposition with the ReLU function](https://mlsanity.com/paper/2305.08687/) (2023)
- [Convergence and complexity of block majorization-minimization for constrained block-Riemannian optimization](https://mlsanity.com/paper/2312.10330/) (2023)
- [Improving the robustness of neural ODEs with minimal weight perturbation](https://mlsanity.com/paper/2501.10740/) (2025)
- [An extrapolated and provably convergent algorithm for nonlinear matrix decomposition with the ReLU function](https://mlsanity.com/paper/2503.23832/) (2025)
- [When big data actually are low-rank, or entrywise approximation of certain function-generated matrices](https://mlsanity.com/paper/2407.03250/) (2024)
- [Efficient algorithms for the Hadamard decomposition](https://mlsanity.com/paper/2504.13633/) (2025)
- [Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositions](https://mlsanity.com/paper/2512.17473/) (2025)
- [Rank-1 matrix differential equations for structured eigenvalue optimization](https://mlsanity.com/paper/2206.09338/) (2022)
- [Matrix nearness problems and eigenvalue optimization](https://mlsanity.com/paper/2503.14750/) (2025)

## Built on by (most cited first, 2 of 2)

- [Riemannian Optimization for Hadamard Products of Low-Rank Matrices](https://mlsanity.com/paper/2606.01216/) (2026)
- [On the Complexity of Low-Rank Matrix Signing and Entrywise Power Matrix Factorization](https://mlsanity.com/paper/2607.04875/) (2026)

## Abstract (arXiv)

Given a matrix X, and two ranks r1 and r2, the Hadamard decomposition (HD) looks for two low-rank matrices, X1 of rank r1 and X2 of rank r2, both of the same size as X, such that X≈ X1circ X2, where circ is the Hadamard (element-wise) product. In most cases, HD is more expressive than standard low-rank approximations such as the truncated singular value decomposition (TSVD), as it can represent higher-rank matrices with the same number of parameters; this is because the rank of X1 circ X2 is generically equal to r1 r2. In this paper, we first present some theoretical insights for HD, in particular a useful reformulation X≈ WH^top where W and H have r1 r2 columns and belong to certain manifolds. These allow us to develop three new algorithms for computing HD. The first one uses the representation X≈ X1circ X2 and relies on the Manopt toolbox. The other two rely on the reformulation X≈ WH^top: one is a block projected gradient method, and the other is a manifold-based gradient descent algorithm that does not require projection onto the feasible set. The last two algorithms are particularly effective for handling large sparse data. We also propose new initializations that allow us to improve the accuracy of the HD. We compare our algorithms and initialization strategies with the TSVD and with the state of the art. Numerical results show that the new methods are efficient and competitive on both synthetic and real data.