# Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies

arXiv: https://arxiv.org/abs/1808.06508 · HTML: https://mlsanity.com/paper/1808.06508/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2018.

## Built on by (most cited first, 30 of 43)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Self-Supervised Models are Continual Learners](https://mlsanity.com/paper/2112.04215/) (2021)
- [Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations](https://mlsanity.com/paper/2103.06342/) (2021)
- [Consistency Regularization for Variational Auto-Encoders](https://mlsanity.com/paper/2105.14859/) (2021)
- [Modular Lifelong Reinforcement Learning via Neural Composition](https://mlsanity.com/paper/2207.00429/) (2022)
- [The Challenges of Continuous Self-Supervised Learning](https://mlsanity.com/paper/2203.12710/) (2022)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [Lifelong Teacher-Student Network Learning](https://mlsanity.com/paper/2107.04689/) (2021)
- [A soft nearest-neighbor framework for continual semi-supervised learning](https://mlsanity.com/paper/2212.05102/) (2022)
- [Task-agnostic Continual Learning with Hybrid Probabilistic Models](https://mlsanity.com/paper/2106.12772/) (2021)
- [SCALE: Online Self-Supervised Lifelong Learning without Prior Knowledge](https://mlsanity.com/paper/2208.11266/) (2022)
- [Exploring Continual Learning of Diffusion Models](https://mlsanity.com/paper/2303.15342/) (2023)
- [Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions](https://mlsanity.com/paper/2203.06852/) (2022)
- [SPeCiaL: Self-Supervised Pretraining for Continual Learning](https://mlsanity.com/paper/2106.09065/) (2021)
- [An Empirical Study on Disentanglement of Negative-free Contrastive Learning](https://mlsanity.com/paper/2206.04756/) (2022)
- [Continual Variational Autoencoder Learning via Online Cooperative Memorization](https://mlsanity.com/paper/2207.10131/) (2022)
- [A Topology-aware Graph Coarsening Framework for Continual Graph Learning](https://mlsanity.com/paper/2401.03077/) (2024)
- [Lifelong Mixture of Variational Autoencoders](https://mlsanity.com/paper/2107.04694/) (2021)
- [TAME: Task Agnostic Continual Learning using Multiple Experts](https://mlsanity.com/paper/2210.03869/) (2022)
- [Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process](https://mlsanity.com/paper/2108.12278/) (2021)
- [Multiband VAE: Latent Space Alignment for Knowledge Consolidation in Continual Learning](https://mlsanity.com/paper/2106.12196/) (2021)
- [A Procedural World Generation Framework for Systematic Evaluation of Continual Learning](https://mlsanity.com/paper/2106.02585/) (2021)
- [Lifelong Generative Modelling Using Dynamic Expansion Graph Model](https://mlsanity.com/paper/2112.08370/) (2021)
- [Thalamus: a brain-inspired algorithm for biologically-plausible continual learning and disentangled representations](https://mlsanity.com/paper/2205.11713/) (2022)
- [Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning](https://mlsanity.com/paper/2403.18266/) (2024)
- [An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning](https://mlsanity.com/paper/2211.16780/) (2022)
- [Class-Prototype Conditional Diffusion Model with Gradient Projection for Continual Learning](https://mlsanity.com/paper/2312.06710/) (2023)
- [Boxhead: A Dataset for Learning Hierarchical Representations](https://mlsanity.com/paper/2110.03628/) (2021)
- [Masked Autoencoders are Efficient Continual Federated Learners](https://mlsanity.com/paper/2306.03542/) (2023)

## Abstract (arXiv)

Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning from piece-wise stationary visual data: Variational Autoencoder with Shared Embeddings (VASE). Based on the Minimum Description Length principle, VASE automatically detects shifts in the data distribution and allocates spare representational capacity to new knowledge, while simultaneously protecting previously learnt representations from catastrophic forgetting. Our approach encourages the learnt representations to be disentangled, which imparts a number of desirable properties: VASE can deal sensibly with ambiguous inputs, it can enhance its own representations through imagination-based exploration, and most importantly, it exhibits semantically meaningful sharing of latents between different datasets. Compared to baselines with entangled representations, our approach is able to reason beyond surface-level statistics and perform semantically meaningful cross-domain inference.