# On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset

arXiv: https://arxiv.org/abs/1906.03292 · HTML: https://mlsanity.com/paper/1906.03292/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2019.

## Built on by (most cited first, 30 of 100)

- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [D'ya like DAGs? A Survey on Structure Learning and Causal Discovery](https://mlsanity.com/paper/2103.02582/) (2021)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Concept-based Explainable Artificial Intelligence: A Survey](https://mlsanity.com/paper/2312.12936/) (2023)
- [Generalization and Robustness Implications in Object-Centric Learning](https://mlsanity.com/paper/2107.00637/) (2021)
- [SODA: Bottleneck Diffusion Models for Representation Learning](https://mlsanity.com/paper/2311.17901/) (2023)
- [Multi-View Causal Representation Learning with Partial Observability](https://mlsanity.com/paper/2311.04056/) (2023)
- [GlanceNets: Interpretabile, Leak-proof Concept-based Models](https://mlsanity.com/paper/2205.15612/) (2022)
- [Weakly Supervised Representation Learning with Sparse Perturbations](https://mlsanity.com/paper/2206.01101/) (2022)
- [Visual Representation Learning Does Not Generalize Strongly Within the Same Domain](https://mlsanity.com/paper/2107.08221/) (2021)
- [Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language](https://mlsanity.com/paper/2210.00482/) (2022)
- [Relating Graph Neural Networks to Structural Causal Models](https://mlsanity.com/paper/2109.04173/) (2021)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [Disentanglement of Correlated Factors via Hausdorff Factorized Support](https://mlsanity.com/paper/2210.07347/) (2022)
- [Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View](https://mlsanity.com/paper/2102.10543/) (2021)
- [PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning](https://mlsanity.com/paper/2308.03977/) (2023)
- [DisDiff: Unsupervised Disentanglement of Diffusion Probabilistic Models](https://mlsanity.com/paper/2301.13721/) (2023)
- [DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices](https://mlsanity.com/paper/2303.13325/) (2023)
- [Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement](https://mlsanity.com/paper/2402.09712/) (2024)
- [Group Robust Classification Without Any Group Information](https://mlsanity.com/paper/2310.18555/) (2023)
- [DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability](https://mlsanity.com/paper/2210.00364/) (2022)
- [On Causally Disentangled Representations](https://mlsanity.com/paper/2112.05746/) (2021)
- [From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modeling](https://mlsanity.com/paper/2310.11011/) (2023)
- [Local Disentanglement in Variational Auto-Encoders Using Jacobian L1 Regularization](https://mlsanity.com/paper/2106.02923/) (2021)
- [Fair Representation: Guaranteeing Approximate Multiple Group Fairness for Unknown Tasks](https://mlsanity.com/paper/2109.00545/) (2021)
- [The Robustness Limits of SoTA Vision Models to Natural Variation](https://mlsanity.com/paper/2210.13604/) (2022)
- [Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddings](https://mlsanity.com/paper/2310.18777/) (2023)
- [Visual Concepts Tokenization](https://mlsanity.com/paper/2205.10093/) (2022)

## Abstract (arXiv)

Learning meaningful and compact representations with disentangled semantic aspects is considered to be of key importance in representation learning. Since real-world data is notoriously costly to collect, many recent state-of-the-art disentanglement models have heavily relied on synthetic toy data-sets. In this paper, we propose a novel data-set which consists of over one million images of physical 3D objects with seven factors of variation, such as object color, shape, size and position. In order to be able to control all the factors of variation precisely, we built an experimental platform where the objects are being moved by a robotic arm. In addition, we provide two more datasets which consist of simulations of the experimental setup. These datasets provide for the first time the possibility to systematically investigate how well different disentanglement methods perform on real data in comparison to simulation, and how simulated data can be leveraged to build better representations of the real world. We provide a first experimental study of these questions and our results indicate that learned models transfer poorly, but that model and hyperparameter selection is an effective means of transferring information to the real world.