# Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.00848 · HTML: https://mlsanity.com/paper/2107.00848/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 60 references that are in the map.
- First posted on arXiv 2 Jul 2021.

## Builds on (most cited first, 30 of 46)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [CLEVRER: CoLlision Events for Video REpresentation and Reasoning](https://mlsanity.com/paper/1910.01442/) (2019)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents](https://mlsanity.com/paper/1709.06009/) (2017)
- [BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning](https://mlsanity.com/paper/1810.08272/) (2018)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Contrastive Learning of Structured World Models](https://mlsanity.com/paper/1911.12247/) (2019)
- [A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms](https://mlsanity.com/paper/1901.10912/) (2019)
- [Physion: Evaluating Physical Prediction from Vision in Humans and Machines](https://mlsanity.com/paper/2106.08261/) (2021)
- [PHYRE: A New Benchmark for Physical Reasoning](https://mlsanity.com/paper/1908.05656/) (2019)
- [CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning](https://mlsanity.com/paper/1910.04744/) (2019)
- [Behaviour Suite for Reinforcement Learning](https://mlsanity.com/paper/1908.03568/) (2019)
- [Learning Neural Causal Models from Unknown Interventions](https://mlsanity.com/paper/1910.01075/) (2019)
- [Entity Abstraction in Visual Model-Based Reinforcement Learning](https://mlsanity.com/paper/1910.12827/) (2019)
- [CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning](https://mlsanity.com/paper/2010.04296/) (2020)

## Built on by (most cited first, 30 of 53)

- [Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA](https://mlsanity.com/paper/2107.10098/) (2021)
- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [Learning to Induce Causal Structure](https://mlsanity.com/paper/2204.04875/) (2022)
- [Causal Dynamics Learning for Task-Independent State Abstraction](https://mlsanity.com/paper/2206.13452/) (2022)
- [An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learning](https://mlsanity.com/paper/2302.04419/) (2023)
- [Is a Modular Architecture Enough?](https://mlsanity.com/paper/2206.02713/) (2022)
- [Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning](https://mlsanity.com/paper/2207.09081/) (2022)
- [Identifying Weight-Variant Latent Causal Models](https://mlsanity.com/paper/2208.14153/) (2022)
- [Causal-JEPA: Learning World Models through Object-Level Latent Masking](https://mlsanity.com/paper/2602.11389/) (2026)
- [On Neural Architecture Inductive Biases for Relational Tasks](https://mlsanity.com/paper/2206.05056/) (2022)
- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [Identifiable Latent Polynomial Causal Models Through the Lens of Change](https://mlsanity.com/paper/2310.15580/) (2023)
- [Explainable Reinforcement Learning via a Causal World Model](https://mlsanity.com/paper/2305.02749/) (2023)
- [Masked Images Are Counterfactual Samples for Robust Fine-tuning](https://mlsanity.com/paper/2303.03052/) (2023)
- [Zero-Shot Object-Centric Representation Learning](https://mlsanity.com/paper/2408.09162/) (2024)
- [Discrete Key-Value Bottleneck](https://mlsanity.com/paper/2207.11240/) (2022)
- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [Unlocking Slot Attention by Changing Optimal Transport Costs](https://mlsanity.com/paper/2301.13197/) (2023)
- [Towards Understanding How Machines Can Learn Causal Overhypotheses](https://mlsanity.com/paper/2206.08353/) (2022)
- [A Review of the Role of Causality in Developing Trustworthy AI Systems](https://mlsanity.com/paper/2302.06975/) (2023)
- [Neurosymbolic Grounding for Compositional World Models](https://mlsanity.com/paper/2310.12690/) (2023)
- [The Role of Pretrained Representations for the OOD Generalization of Reinforcement Learning Agents](https://mlsanity.com/paper/2107.05686/) (2021)
- [Compositional Attention: Disentangling Search and Retrieval](https://mlsanity.com/paper/2110.09419/) (2021)
- [Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning](https://mlsanity.com/paper/2406.03234/) (2024)
- [Test-time Adaptation with Slot-Centric Models](https://mlsanity.com/paper/2203.11194/) (2022)
- [On the Generalization and Adaption Performance of Causal Models](https://mlsanity.com/paper/2206.04620/) (2022)
- [Intrinsic Physical Concepts Discovery with Object-Centric Predictive Models](https://mlsanity.com/paper/2303.01869/) (2023)
- [Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior](https://mlsanity.com/paper/2305.19550/) (2023)
- [Learning Causal Overhypotheses through Exploration in Children and Computational Models](https://mlsanity.com/paper/2202.10430/) (2022)
- [Cycle Consistency Driven Object Discovery](https://mlsanity.com/paper/2306.02204/) (2023)

## Abstract (arXiv)

Inducing causal relationships from observations is a classic problem in machine learning. Most work in causality starts from the premise that the causal variables themselves are observed. However, for AI agents such as robots trying to make sense of their environment, the only observables are low-level variables like pixels in images. To generalize well, an agent must induce high-level variables, particularly those which are causal or are affected by causal variables. A central goal for AI and causality is thus the joint discovery of abstract representations and causal structure. However, we note that existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs which are impossible to manipulate parametrically (e.g., number of nodes, sparsity, causal chain length, etc.). In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them. In order to systematically probe the ability of methods to identify these variables and structures, we design a suite of benchmarking RL environments. We evaluate various representation learning algorithms from the literature and find that explicitly incorporating structure and modularity in models can help causal induction in model-based reinforcement learning.