# Disentangled Representation Learning and Generation with Manifold Optimization

arXiv: https://arxiv.org/abs/2006.07046 · HTML: https://mlsanity.com/paper/2006.07046/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 15 of 15)

- [Unsupervised Energy-based Out-of-distribution Detection using Stiefel-Restricted Kernel Machine](https://mlsanity.com/paper/2102.08443/) (2021)
- [TRKM: Twin Restricted Kernel Machines for Classification and Regression](https://mlsanity.com/paper/2502.15759/) (2025)
- [Tensor-based Multi-view Spectral Clustering via Shared Latent Space](https://mlsanity.com/paper/2207.11559/) (2022)
- [One Class Restricted Kernel Machines](https://mlsanity.com/paper/2502.10443/) (2025)
- [Unsupervised Neighborhood Propagation Kernel Layers for Semi-supervised Node Classification](https://mlsanity.com/paper/2301.13764/) (2023)
- [Randomized based restricted kernel machine for hyperspectral image classification](https://mlsanity.com/paper/2503.05837/) (2025)
- [Multi-view Kernel PCA for Time series Forecasting](https://mlsanity.com/paper/2301.09811/) (2023)
- [HeNCler: Node Clustering in Heterophilous Graphs via Learned Asymmetric Similarity](https://mlsanity.com/paper/2405.17050/) (2024)
- [Duality in Multi-View Restricted Kernel Machines](https://mlsanity.com/paper/2305.17251/) (2023)
- [Transferring disentangled representations: bridging the gap between synthetic and real images](https://mlsanity.com/paper/2409.18017/) (2024)
- [This Looks Distinctly Like That: Grounding Interpretable Recognition in Stiefel Geometry against Neural Collapse](https://mlsanity.com/paper/2603.08374/) (2026)
- [A Dual Formulation for Probabilistic Principal Component Analysis](https://mlsanity.com/paper/2307.10078/) (2023)
- [Generative Kernel Spectral Clustering](https://mlsanity.com/paper/2502.02185/) (2025)
- [Machine Unlearning in Low-Dimensional Feature Subspace](https://mlsanity.com/paper/2601.22456/) (2026)
- [Distilling Latent Manifolds: Resolution Extrapolation by Variational Autoencoders](https://mlsanity.com/paper/2603.14536/) (2026)

## Abstract (arXiv)

Disentanglement is a useful property in representation learning which increases the interpretability of generative models such as Variational autoencoders (VAE), Generative Adversarial Models, and their many variants. Typically in such models, an increase in disentanglement performance is traded-off with generation quality. In the context of latent space models, this work presents a representation learning framework that explicitly promotes disentanglement by encouraging orthogonal directions of variations. The proposed objective is the sum of an autoencoder error term along with a Principal Component Analysis reconstruction error in the feature space. This has an interpretation of a Restricted Kernel Machine with the eigenvector matrix-valued on the Stiefel manifold. Our analysis shows that such a construction promotes disentanglement by matching the principal directions in the latent space with the directions of orthogonal variation in data space. In an alternating minimization scheme, we use Cayley ADAM algorithm - a stochastic optimization method on the Stiefel manifold along with the ADAM optimizer. Our theoretical discussion and various experiments show that the proposed model improves over many VAE variants in terms of both generation quality and disentangled representation learning.