mlsanityData to 14 Oct 2026

Learning Causal Dynamics Models in Object-Oriented Environments

arXiv 2405.12615 ↗First posted 21 May 2024cs.LGcited by 3

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Causal dynamics models (CDMs) have demonstrated significant potential in addressing various challenges in reinforcement learning. To learn CDMs, recent studies have performed causal discovery to capture the causal dependencies among environmental variables. However, the learning of CDMs is still confined to small-scale environments due to computational complexity and sample efficiency constraints. This paper aims to extend CDMs to large-scale object-oriented environments, which consist of a multitude of objects classified into different categories. We introduce the Object-Oriented CDM (OOCDM) that shares causalities and parameters among objects belonging to the same class. Furthermore, we propose a learning method for OOCDM that enables it to adapt to a varying number of objects. Experiments on large-scale tasks indicate that OOCDM outperforms existing CDMs in terms of causal discovery, prediction accuracy, generalization, and computational efficiency.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 57 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

30 of 57 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Object-Centric Learning with Slot Attentioncited by 876
  3. 2019Contrastive Learning of Structured World Modelscited by 209
  4. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  5. 2020Invariant Causal Prediction for Block MDPscited by 105
  6. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  7. 2018Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Searchcited by 82
  8. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 65
  9. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  10. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  11. 2020Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systemscited by 48
  12. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 48
18 more
  1. 2023A Survey on Causal Reinforcement Learningcited by 45
  2. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 44
  3. 2014Near-optimal Reinforcement Learning in Factored MDPscited by 41
  4. 2020Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modulescited by 38
  5. 2018Deconfounding Reinforcement Learning in Observational Settingscited by 36
  6. 2022Is a Modular Architecture Enough?cited by 36
  7. 2021Instrumental Variable Value Iteration for Causal Offline Reinforcement Learningcited by 35
  8. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 34
  9. 2019Causal Induction from Visual Observations for Goal Directed Taskscited by 31
  10. 2022Factored Adaptation for Non-Stationary Reinforcement Learningcited by 30
  11. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 24
  12. 2022Policy Architectures for Compositional Generalization in Controlcited by 23
  13. 2023Explainable Reinforcement Learning via a Causal World Modelcited by 22
  14. 2022Offline Reinforcement Learning with Causal Structured World Modelscited by 19
  15. 2018Fast Conditional Independence Test for Vector Variables with Large Sample Sizescited by 16
  16. 2018Object-Oriented Dynamics Predictorcited by 15
  17. 2022A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learningcited by 15
  18. 2019Object-Oriented Dynamics Learning through Multi-Level Abstractioncited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  2. 2025Curious Causality-Seeking Agents Learn Meta Causal Worldcited by 4
  3. 2024Systems with Switching Causal Relations: A Meta-Causal Perspectivecited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Better Decisions through the Right Causal World Model
  2. 2024Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning
  3. 2025Causal Process Models: Reframing Dynamic Causal Graph Discovery as a Reinforcement Learning Problem
  4. 2025Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting
  5. 2025Towards Causal Model-Based Policy Optimization
  6. 2024Causal Reinforcement Learning for Optimisation of Robot Dynamics in Unknown Environments
  7. 2024Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning
  8. 2025Learning Local Causal World Models with State Space Models and Attention