# A Flexible and Efficient Algorithmic Framework for Constrained Matrix and Tensor Factorization

arXiv: https://arxiv.org/abs/1506.04209 · HTML: https://mlsanity.com/paper/1506.04209/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2015.

## Built on by (most cited first, 30 of 30)

- [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022)
- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [An AO-ADMM approach to constraining PARAFAC2 on all modes](https://mlsanity.com/paper/2110.01278/) (2021)
- [Graph-based Neural Acceleration for Nonnegative Matrix Factorization](https://mlsanity.com/paper/2202.00264/) (2022)
- [A Provable Splitting Approach for Symmetric Nonnegative Matrix Factorization](https://mlsanity.com/paper/2301.10499/) (2023)
- [Quantization Aware Factorization for Deep Neural Network Compression](https://mlsanity.com/paper/2308.04595/) (2023)
- [Stochastic Mirror Descent for Low-Rank Tensor Decomposition Under Non-Euclidean Losses](https://mlsanity.com/paper/2104.14562/) (2021)
- [PARAFAC2-based Coupled Matrix and Tensor Factorizations](https://mlsanity.com/paper/2210.13054/) (2022)
- [Alternating Direction Method of Multipliers for Nonlinear Matrix Decompositions](https://mlsanity.com/paper/2512.17473/) (2025)
- [PARAFAC2 AO-ADMM: Constraints in all modes](https://mlsanity.com/paper/2102.02087/) (2021)
- [Efficient Multidimensional Functional Data Analysis Using Marginal Product Basis Systems](https://mlsanity.com/paper/2107.14728/) (2021)
- [MASIL: Towards Maximum Separable Class Representation for Few Shot Class Incremental Learning](https://mlsanity.com/paper/2304.05362/) (2023)
- [A Time-aware tensor decomposition for tracking evolving patterns](https://mlsanity.com/paper/2308.07126/) (2023)
- [Low-Rank Tensors for Multi-Dimensional Markov Models](https://mlsanity.com/paper/2411.02098/) (2024)
- [Identifiability of Nonnegative Tucker Decompositions -- Part I: Theory](https://mlsanity.com/paper/2505.12713/) (2025)
- [SWoTTeD: An Extension of Tensor Decomposition to Temporal Phenotyping](https://mlsanity.com/paper/2310.01201/) (2023)
- [PARAFAC2-based Coupled Matrix and Tensor Factorizations with Constraints](https://mlsanity.com/paper/2406.12338/) (2024)
- [tPARAFAC2: Tracking evolving patterns in (incomplete) temporal data](https://mlsanity.com/paper/2407.01356/) (2024)
- [Toward Temporal Causal Representation Learning with Tensor Decomposition](https://mlsanity.com/paper/2507.14126/) (2025)
- [An Exterior Method for Nonnegative Matrix Factorization](https://mlsanity.com/paper/2605.19325/) (2026)
- [Dictionary-based Low-Rank Approximations and the Mixed Sparse Coding problem](https://mlsanity.com/paper/2111.12399/) (2021)
- [Learning Multivariate CDFs and Copulas using Tensor Factorization](https://mlsanity.com/paper/2210.07132/) (2022)
- [Scalable and Robust Tensor Ring Decomposition for Large-scale Data](https://mlsanity.com/paper/2305.09044/) (2023)
- [MultiHU-TD: Multifeature Hyperspectral Unmixing Based on Tensor Decomposition](https://mlsanity.com/paper/2310.03860/) (2023)
- [Interpretable factorization of clinical questionnaires to identify latent factors of psychopathology](https://mlsanity.com/paper/2312.07762/) (2023)
- [ADMM-MM Algorithm for General Tensor Decomposition](https://mlsanity.com/paper/2312.11763/) (2023)
- [Multi-View Spectral Clustering for Graphs with Multiple View Structures](https://mlsanity.com/paper/2501.11422/) (2025)
- [dCMF: Learning interpretable evolving patterns from temporal multiway data](https://mlsanity.com/paper/2502.19367/) (2025)
- [Generalized Nonnegative Structured Kruskal Tensor Regression](https://mlsanity.com/paper/2509.19900/) (2025)
- [Sparse Network Inference under Imperfect Detection and its Application to Ecological Networks](https://mlsanity.com/paper/2604.18820/) (2026)

## Abstract (arXiv)

We propose a general algorithmic framework for constrained matrix and tensor factorization, which is widely used in signal processing and machine learning. The new framework is a hybrid between alternating optimization (AO) and the alternating direction method of multipliers (ADMM): each matrix factor is updated in turn, using ADMM, hence the name AO-ADMM. This combination can naturally accommodate a great variety of constraints on the factor matrices, and almost all possible loss measures for the fitting. Computation caching and warm start strategies are used to ensure that each update is evaluated efficiently, while the outer AO framework exploits recent developments in block coordinate descent (BCD)-type methods which help ensure that every limit point is a stationary point, as well as faster and more robust convergence in practice. Three special cases are studied in detail: non-negative matrix/tensor factorization, constrained matrix/tensor completion, and dictionary learning. Extensive simulations and experiments with real data are used to showcase the effectiveness and broad applicability of the proposed framework.