# Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces

arXiv: https://arxiv.org/abs/2607.05653 · HTML: https://mlsanity.com/paper/2607.05653/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Lagging Inference Networks and Posterior Collapse in Variational Autoencoders](https://mlsanity.com/paper/1901.05534/) (2019)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Variational AutoEncoder For Regression: Application to Brain Aging Analysis](https://mlsanity.com/paper/1904.05948/) (2019)
- [Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey](https://mlsanity.com/paper/2101.00734/) (2021)
- [A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold](https://mlsanity.com/paper/2302.08210/) (2023)
- [Hebbian learning with gradients: Hebbian convolutional neural networks with modern deep learning frameworks](https://mlsanity.com/paper/2107.01729/) (2021)
- [Generalized BackPropagation, 'Etude De Cas: Orthogonality](https://mlsanity.com/paper/1611.05927/) (2016)
- [PCAAE: Principal Component Analysis Autoencoder for organising the latent space of generative networks](https://mlsanity.com/paper/2006.07827/) (2020)
- [Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing](https://mlsanity.com/paper/2310.15440/) (2023)
- [Beta-VAE Reproducibility: Challenges and Extensions](https://mlsanity.com/paper/2112.14278/) (2021)
- [Learnable Faster Kernel-PCA for Nonlinear Fault Detection: Deep Autoencoder-Based Realization](https://mlsanity.com/paper/2112.04193/) (2021)
- [Principal Orthogonal Latent Components Analysis (POLCA Net)](https://mlsanity.com/paper/2410.07289/) (2024)

## Abstract (arXiv)

Principal Component Analysis or PCA-like properties (orthogonality, variance ranking) are seldom realized in deep autoencoder architectures. In this work, we present ODIN (Orthogonal Dendritic Intrinsic Network), a novel autoencoder architecture that recovers PCA-like latent structure in a fully non-linear regime. By incorporating a set of geometric constraints directly into the training objective, ODIN encourages latent dimensions to be mutually orthogonal and ordered by explained variance, mirroring the interpretable decomposition of PCA while retaining the expressive power of deep networks. We provide theoretical grounding for these constraints and demonstrate their compatibility with standard encoder-decoder frameworks. We also establish empirical results for both synthetic and real world datasets, establishing a principled path toward interpretable, structured feature learning and dimensionality reduction.