# Efficient algorithms for the Hadamard decomposition

arXiv: https://arxiv.org/abs/2504.13633 · HTML: https://mlsanity.com/paper/2504.13633/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 14 references that are in the map.
- First posted on arXiv 18 Apr 2025.

## Builds on (most cited first, 2 of 2)

- [FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2108.06098/) (2021)
- [Accelerating Nonnegative Matrix Factorization Algorithms using Extrapolation](https://mlsanity.com/paper/1805.06604/) (2018)

## Built on by (most cited first, 3 of 3)

- [Low-Rank Adaptation Redux for Large Models](https://mlsanity.com/paper/2604.21905/) (2026)
- [Manifold-based Algorithms for the Hadamard Decomposition](https://mlsanity.com/paper/2605.28980/) (2026)
- [Riemannian Optimization for Hadamard Products of Low-Rank Matrices](https://mlsanity.com/paper/2606.01216/) (2026)

## Abstract (arXiv)

The Hadamard decomposition is a powerful technique for data analysis and matrix compression, which decomposes a given matrix into the element-wise product of two or more low-rank matrices. In this paper, we develop an efficient algorithm to solve this problem, leveraging an alternating optimization approach that decomposes the global non-convex problem into a series of convex sub-problems. To improve performance, we explore advanced initialization strategies inspired by the singular value decomposition (SVD) and incorporate acceleration techniques by introducing momentum-based updates. Beyond optimizing the two-matrix case, we also extend the Hadamard decomposition framework to support more than two low-rank matrices, enabling approximations with higher effective ranks while preserving computational efficiency. Finally, we conduct extensive experiments to compare our method with the existing gradient descent-based approaches for the Hadamard decomposition and with traditional low-rank approximation techniques. The results highlight the effectiveness of our proposed method across diverse datasets.