# Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularization

arXiv: https://arxiv.org/abs/2106.02923 · HTML: https://mlsanity.com/paper/2106.02923/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 25 of 25)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [VAE with a VampPrior](https://mlsanity.com/paper/1705.07120/) (2017)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [k-Sparse Autoencoders](https://mlsanity.com/paper/1312.5663/) (2013)
- [Hyperspherical Variational Auto-Encoders](https://mlsanity.com/paper/1804.00891/) (2018)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs](https://mlsanity.com/paper/1901.07017/) (2019)
- [Learning Deep Disentangled Embeddings with the F-Statistic Loss](https://mlsanity.com/paper/1802.05312/) (2018)
- [On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset](https://mlsanity.com/paper/1906.03292/) (2019)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [Bias and Generalization in Deep Generative Models: An Empirical Study](https://mlsanity.com/paper/1811.03259/) (2018)
- [Resampled Priors for Variational Autoencoders](https://mlsanity.com/paper/1810.11428/) (2018)
- [On Implicit Regularization in β-VAEs](https://mlsanity.com/paper/2002.00041/) (2020)
- [Learning Flat Latent Manifolds with VAEs](https://mlsanity.com/paper/2002.04881/) (2020)
- [Gradient Regularization Improves Accuracy of Discriminative Models](https://mlsanity.com/paper/1712.09936/) (2017)
- [Independent Subspace Analysis for Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1909.05063/) (2019)
- [Epitomic Variational Graph Autoencoder](https://mlsanity.com/paper/2004.01468/) (2020)
- [Improved Training of Sparse Coding Variational Autoencoder via Weight Normalization](https://mlsanity.com/paper/2101.09453/) (2021)

## Built on by (most cited first, 17 of 17)

- [On the Identifiability of Nonlinear ICA: Sparsity and Beyond](https://mlsanity.com/paper/2206.07751/) (2022)
- [GlanceNets: Interpretabile, Leak-proof Concept-based Models](https://mlsanity.com/paper/2205.15612/) (2022)
- [Identifiable Deep Generative Models via Sparse Decoding](https://mlsanity.com/paper/2110.10804/) (2021)
- [Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learning](https://mlsanity.com/paper/2309.07742/) (2023)
- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [Jacobian Regularizer-based Neural Granger Causality](https://mlsanity.com/paper/2405.08779/) (2024)
- [PGODE: Towards High-quality System Dynamics Modeling](https://mlsanity.com/paper/2311.06554/) (2023)
- [3D Generative Model Latent Disentanglement via Local Eigenprojection](https://mlsanity.com/paper/2302.12798/) (2023)
- [The Gaussian Discriminant Variational Autoencoder (GdVAE): A Self-Explainable Model with Counterfactual Explanations](https://mlsanity.com/paper/2409.12952/) (2024)
- [Causal Representation Learning in Temporal Data via Single-Parent Decoding](https://mlsanity.com/paper/2410.07013/) (2024)
- [Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability](https://mlsanity.com/paper/2510.17040/) (2025)
- [Mechanistic Independence: A Principle for Identifiable Disentangled Representations](https://mlsanity.com/paper/2509.22196/) (2025)
- [Nonparametric Identification of Latent Concepts](https://mlsanity.com/paper/2510.00136/) (2025)
- [Ensuring Topological Data-Structure Preservation under Autoencoder Compression due to Latent Space Regularization in Gauss--Legendre nodes](https://mlsanity.com/paper/2309.08228/) (2023)
- [Subgroup Discovery in Unstructured Data](https://mlsanity.com/paper/2207.07781/) (2022)
- [Layer-Specific Lipschitz Modulation for Fault-Tolerant Multimodal Representation Learning](https://mlsanity.com/paper/2603.25103/) (2026)
- [Diverse Dictionary Learning](https://mlsanity.com/paper/2604.17568/) (2026)

## Abstract (arXiv)

There have been many recent advances in representation learning; however, unsupervised representation learning can still struggle with model identification issues related to rotations of the latent space. Variational Auto-Encoders (VAEs) and their extensions such as β-VAEs have been shown to improve local alignment of latent variables with PCA directions, which can help to improve model disentanglement under some conditions. Borrowing inspiration from Independent Component Analysis (ICA) and sparse coding, we propose applying an L1 loss to the VAE's generative Jacobian during training to encourage local latent variable alignment with independent factors of variation in images of multiple objects or images with multiple parts. We demonstrate our results on a variety of datasets, giving qualitative and quantitative results using information theoretic and modularity measures that show our added L1 cost encourages local axis alignment of the latent representation with individual factors of variation.