# Lost in Latent Space: Disentangled Models and the Challenge of Combinatorial Generalisation

arXiv: https://arxiv.org/abs/2204.02283 · HTML: https://mlsanity.com/paper/2204.02283/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Wasserstein Auto-Encoders](https://mlsanity.com/paper/1711.01558/) (2017)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA](https://mlsanity.com/paper/1605.06336/) (2016)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [DARLA: Improving Zero-Shot Transfer in Reinforcement Learning](https://mlsanity.com/paper/1707.08475/) (2017)
- [Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding](https://mlsanity.com/paper/2007.10930/) (2020)
- [Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs](https://mlsanity.com/paper/1901.07017/) (2019)
- [Are Disentangled Representations Helpful for Abstract Visual Reasoning?](https://mlsanity.com/paper/1905.12506/) (2019)
- [On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset](https://mlsanity.com/paper/1906.03292/) (2019)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Unsupervised Model Selection for Variational Disentangled Representation Learning](https://mlsanity.com/paper/1905.12614/) (2019)
- [Commutative Lie Group VAE for Disentanglement Learning](https://mlsanity.com/paper/2106.03375/) (2021)
- [Learning Discrete and Continuous Factors of Data via Alternating Disentanglement](https://mlsanity.com/paper/1905.09432/) (2019)

## Built on by (most cited first, 11 of 11)

- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Neither hype nor gloom do DNNs justice](https://mlsanity.com/paper/2312.05355/) (2023)
- [Disentanglement of Correlated Factors via Hausdorff Factorized Support](https://mlsanity.com/paper/2210.07347/) (2022)
- [Does Data Scaling Lead to Visual Compositional Generalization?](https://mlsanity.com/paper/2507.07102/) (2025)
- [Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Models](https://mlsanity.com/paper/2311.09064/) (2023)
- [Towards Combinatorial Generalization for Catalysts: A Kohn-Sham Charge-Density Approach](https://mlsanity.com/paper/2310.18702/) (2023)
- [Symbolic Disentangled Representations for Images](https://mlsanity.com/paper/2412.19847/) (2024)
- [Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics](https://mlsanity.com/paper/2303.00077/) (2023)
- [Fixing the problems of deep neural networks will require better training data and learning algorithms](https://mlsanity.com/paper/2311.12819/) (2023)
- [Contrastive-Augmented Flow Matching for Style-Content Disentanglement](https://mlsanity.com/paper/2607.12404/) (2026)
- [TPR-Attention for Combinatorial Generalization](https://mlsanity.com/paper/2608.30124/) (2026)

## Abstract (arXiv)

Recent research has shown that generative models with highly disentangled representations fail to generalise to unseen combination of generative factor values. These findings contradict earlier research which showed improved performance in out-of-training distribution settings when compared to entangled representations. Additionally, it is not clear if the reported failures are due to (a) encoders failing to map novel combinations to the proper regions of the latent space or (b) novel combinations being mapped correctly but the decoder/downstream process is unable to render the correct output for the unseen combinations. We investigate these alternatives by testing several models on a range of datasets and training settings. We find that (i) when models fail, their encoders also fail to map unseen combinations to correct regions of the latent space and (ii) when models succeed, it is either because the test conditions do not exclude enough examples, or because excluded generative factors determine independent parts of the output image. Based on these results, we argue that to generalise properly, models not only need to capture factors of variation, but also understand how to invert the generative process that was used to generate the data.