# Non-Negative Stiefel Approximating Flow: Orthogonalish Matrix Optimization for Interpretable Embeddings

arXiv: https://arxiv.org/abs/2511.06425 · HTML: https://mlsanity.com/paper/2511.06425/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 41 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Builds on (most cited first, 8 of 8)

- [Large Batch Optimization for Deep Learning: Training BERT in 76 minutes](https://mlsanity.com/paper/1904.00962/) (2019)
- [Manopt, a Matlab toolbox for optimization on manifolds](https://mlsanity.com/paper/1308.5200/) (2013)
- [boldsymbolλ-Orthogonality Regularization for Compatible Representation Learning](https://mlsanity.com/paper/2509.16664/) (2025)
- [A Frobenius norm regularization method for convolutional kernels to avoid unstable gradient problem](https://mlsanity.com/paper/1907.11235/) (2019)
- [Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs](https://mlsanity.com/paper/2205.08385/) (2022)
- [The Restricted Isometry of ReLU Networks: Generalization through Norm Concentration](https://mlsanity.com/paper/2007.00479/) (2020)
- [Towards Better Orthogonality Regularization with Disentangled Norm in Training Deep CNNs](https://mlsanity.com/paper/2306.09939/) (2023)
- [Group Orthogonalization Regularization For Vision Models Adaptation and Robustness](https://mlsanity.com/paper/2306.10001/) (2023)

## Built on by (most cited first, 1 of 1)

- [Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces](https://mlsanity.com/paper/2610.00751/) (2026)

## Abstract (arXiv)

Interpretable representation learning is a central challenge in modern machine learning, particularly in high-dimensional settings such as neuroimaging, genomics, and text analysis. Current methods often struggle to balance the competing demands of interpretability and model flexibility, limiting their effectiveness in extracting meaningful insights from complex data. We introduce Non-negative Stiefel Approximating Flow (NSA-Flow), a general-purpose matrix estimation framework that unifies ideas from sparse matrix factorization, orthogonalization, and constrained manifold learning. NSA-Flow enforces structured sparsity through a continuous balance between reconstruction fidelity and column-wise decorrelation, parameterized by a single tunable weight. The method operates as a smooth flow near the Stiefel manifold with proximal updates for non-negativity and adaptive gradient control, yielding representations that are simultaneously sparse, stable, and interpretable. Unlike classical regularization schemes, NSA-Flow provides an intuitive geometric mechanism for manipulating sparsity at the level of global structure while simplifying latent features. We demonstrate that the NSA-Flow objective can be optimized smoothly and integrates seamlessly with existing pipelines for dimensionality reduction while improving interpretability and generalization in both simulated and real biomedical data. Empirical validation on the Golub leukemia dataset and in Alzheimer's disease demonstrate that the NSA-Flow constraints can maintain or improve performance over related methods with little additional methodological effort. NSA-Flow offers a scalable, general-purpose tool for interpretable ML, applicable across data science domains.