# Learning Conditional Invariance through Cycle Consistency

arXiv: https://arxiv.org/abs/2111.13185 · HTML: https://mlsanity.com/paper/2111.13185/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 25 Nov 2021.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [Isolating Sources of Disentanglement in Variational Autoencoders](https://mlsanity.com/paper/1802.04942/) (2018)
- [Understanding the Limitations of Variational Mutual Information Estimators](https://mlsanity.com/paper/1910.06222/) (2019)
- [Information Dropout: Learning Optimal Representations Through Noisy Computation](https://mlsanity.com/paper/1611.01353/) (2016)
- [Fader Networks: Manipulating Images by Sliding Attributes](https://mlsanity.com/paper/1706.00409/) (2017)
- [Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations](https://mlsanity.com/paper/1705.08841/) (2017)
- [Beyond Sparsity: Tree Regularization of Deep Models for Interpretability](https://mlsanity.com/paper/1711.06178/) (2017)
- [Learning Latent Subspaces in Variational Autoencoders](https://mlsanity.com/paper/1812.06190/) (2018)
- [Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoders](https://mlsanity.com/paper/1804.10469/) (2018)
- [3DMolNet: A Generative Network for Molecular Structures](https://mlsanity.com/paper/2010.06477/) (2020)
- [Learning Sparse Latent Representations with the Deep Copula Information Bottleneck](https://mlsanity.com/paper/1804.06216/) (2018)
- [Learning Extremal Representations with Deep Archetypal Analysis](https://mlsanity.com/paper/2002.00815/) (2020)
- [Adversarial Information Factorization](https://mlsanity.com/paper/1711.05175/) (2017)
- [On the Difference Between the Information Bottleneck and the Deep Information Bottleneck](https://mlsanity.com/paper/1912.13480/) (2019)
- [Deep Archetypal Analysis](https://mlsanity.com/paper/1901.10799/) (2019)
- [Causal learning with sufficient statistics: an information bottleneck approach](https://mlsanity.com/paper/2010.05375/) (2020)
- [DualDis: Dual-Branch Disentangling with Adversarial Learning](https://mlsanity.com/paper/1906.00804/) (2019)
- [Causal Compression](https://mlsanity.com/paper/1611.00261/) (2016)

## Built on by (most cited first, 2 of 2)

- [Learning Invariances with Generalised Input-Convex Neural Networks](https://mlsanity.com/paper/2204.07009/) (2022)
- [Learning Conditional Invariances through Non-Commutativity](https://mlsanity.com/paper/2402.11682/) (2024)

## Abstract (arXiv)

Identifying meaningful and independent factors of variation in a dataset is a challenging learning task frequently addressed by means of deep latent variable models. This task can be viewed as learning symmetry transformations preserving the value of a chosen property along latent dimensions. However, existing approaches exhibit severe drawbacks in enforcing the invariance property in the latent space. We address these shortcomings with a novel approach to cycle consistency. Our method involves two separate latent subspaces for the target property and the remaining input information, respectively. In order to enforce invariance as well as sparsity in the latent space, we incorporate semantic knowledge by using cycle consistency constraints relying on property side information. The proposed method is based on the deep information bottleneck and, in contrast to other approaches, allows using continuous target properties and provides inherent model selection capabilities. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models with improved invariance properties.