# Clustering by Orthogonal NMF Model and Non-Convex Penalty Optimization

arXiv: https://arxiv.org/abs/1906.00570 · HTML: https://mlsanity.com/paper/1906.00570/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 9 of 9)

- [Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Framework](https://mlsanity.com/paper/2212.01519/) (2022)
- [Differentially Private Federated Clustering over Non-IID Data](https://mlsanity.com/paper/2301.00955/) (2023)
- [Nonnegative Matrix Factorization in Dimensionality Reduction: A Survey](https://mlsanity.com/paper/2405.03615/) (2024)
- [Federated Semi-Supervised Learning with Class Distribution Mismatch](https://mlsanity.com/paper/2111.00010/) (2021)
- [Structured Sparse Non-negative Matrix Factorization with L20-Norm for scRNA-seq Data Analysis](https://mlsanity.com/paper/2104.13171/) (2021)
- [Community detection in multiplex networks based on orthogonal nonnegative matrix tri-factorization](https://mlsanity.com/paper/2205.00626/) (2022)
- [A Support-Set Algorithm for Optimization Problems with Nonnegative and Orthogonal Constraints](https://mlsanity.com/paper/2511.03443/) (2025)
- [GAMMA_FLOW: Guided Analysis of Multi-label spectra by MAtrix Factorization for Lightweight Operational Workflows](https://mlsanity.com/paper/2511.09326/) (2025)
- [Riemannian Difference-of-Convex Optimization for K-Means Clustering](https://mlsanity.com/paper/2609.34310/) (2026)

## Abstract (arXiv)

The non-negative matrix factorization (NMF) model with an additional orthogonality constraint on one of the factor matrices, called the orthogonal NMF (ONMF), has been found a promising clustering model and can outperform the classical K-means. However, solving the ONMF model is a challenging optimization problem because the coupling of the orthogonality and non-negativity constraints introduces a mixed combinatorial aspect into the problem due to the determination of the correct status of the variables (positive or zero). Most of the existing methods directly deal with the orthogonality constraint in its original form via various optimization techniques, but are not scalable for large-scale problems. In this paper, we propose a new ONMF based clustering formulation that equivalently transforms the orthogonality constraint into a set of norm-based non-convex equality constraints. We then apply a non-convex penalty (NCP) approach to add them to the objective as penalty terms, leading to a problem that is efficiently solvable. One smooth penalty formulation and one non-smooth penalty formulation are respectively studied. We build theoretical conditions for the penalized problems to provide feasible stationary solutions to the ONMF based clustering problem, as well as proposing efficient algorithms for solving the penalized problems of the two NCP methods. Experimental results based on both synthetic and real datasets are presented to show that the proposed NCP methods are computationally time efficient, and either match or outperform the existing K-means and ONMF based methods in terms of the clustering performance.