mlsanityData to 14 Oct 2026

Causal Dynamics Learning for Task-Independent State Abstraction

arXiv 2206.13452 ↗First posted 27 Jun 2022cs.LGcited by 65

Not on the map. Why?

Cited by 30 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Learning dynamics models accurately is an important goal for Model-Based Reinforcement Learning (MBRL), but most MBRL methods learn a dense dynamics model which is vulnerable to spurious correlations and therefore generalizes poorly to unseen states. In this paper, we introduce Causal Dynamics Learning for Task-Independent State Abstraction (CDL), which first learns a theoretically proved causal dynamics model that removes unnecessary dependencies between state variables and the action, thus generalizing well to unseen states. A state abstraction can then be derived from the learned dynamics, which not only improves sample efficiency but also applies to a wider range of tasks than existing state abstraction methods. Evaluated on two simulated environments and downstream tasks, both the dynamics model and policies learned by the proposed method generalize well to unseen states and the derived state abstraction improves sample efficiency compared to learning without it.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 27 Jun 2022.

Builds on

30 of 43 references.

References · 30

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  5. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  6. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  7. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  8. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  9. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
  10. 2019Recurrent Independent Mechanismscited by 219
  11. 2019Contrastive Learning of Structured World Modelscited by 209
  12. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
18 more
  1. 2020Invariant Causal Prediction for Block MDPscited by 105
  2. 2018Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Searchcited by 82
  3. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 78
  4. 2020Causal Discovery in Physical Systems from Videoscited by 77
  5. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  6. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
  7. 2020Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentationcited by 59
  8. 2021Learning Task Informed Abstractionscited by 58
  9. 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
  10. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  11. 2020Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systemscited by 48
  12. 2019Sub-policy Adaptation for Hierarchical Reinforcement Learningcited by 47
  13. 2020On the model-based stochastic value gradient for continuous reinforcement learningcited by 47
  14. 2019Causal Induction from Visual Observations for Goal Directed Taskscited by 31
  15. 2019Learning Causal State Representations of Partially Observable Environmentscited by 30
  16. 2021Model-Invariant State Abstractions for Model-Based Reinforcement Learningcited by 18
  17. 2020Resolving Spurious Correlations in Causal Models of Environments via Interventionscited by 14
  18. 2020Intervention Design for Effective Sim2Real Transfercited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 11; 2024: 18; 2025: 19; 2026: 12.

65 papers.

Sort:

By area:

  1. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 45
  2. 2023A Survey on Causal Reinforcement Learningcited by 45
  3. 2022MoCoDA: Model-based Counterfactual Data Augmentationcited by 44
  4. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 34
  5. 2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
  6. 2023Causal Reinforcement Learning: A Surveycited by 23
  7. 2023Explainable Reinforcement Learning via a Causal World Modelcited by 22
  8. 2023Causal Policy Gradient for Whole-Body Mobile Manipulationcited by 21
  9. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 21
  10. 2023Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approachcited by 19
  11. 2022Agent-Controller Representations: Principled Offline RL with Rich Exogenous Informationcited by 16
  12. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
38 more
  1. 2023A Review of the Role of Causality in Developing Trustworthy AI Systemscited by 14
  2. 2024Causal Action Influence Aware Counterfactual Data Augmentationcited by 12
  3. 2024Building Minimal and Reusable Causal State Abstractions for Reinforcement Learningcited by 11
  4. 2024Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learningcited by 11
  5. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  6. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  7. 2023ELDEN: Exploration via Local Dependenciescited by 8
  8. 2025SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Trainingcited by 7
  9. 2026Factored Latent Action World Modelscited by 7
  10. 2023Granger Causal Interaction Skill Chainscited by 6
  11. 2023MACCA: Offline Multi-agent Reinforcement Learning with Causal Credit Assignmentcited by 6
  12. 2025Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learningcited by 6
  13. 2025Towards Empowerment Gain through Causal Structure Learning in Model-Based RLcited by 5
  14. 2024Automated Discovery of Functional Actual Causes in Complex Environmentscited by 4
  15. 2025Curious Causality-Seeking Agents Learn Meta Causal Worldcited by 4
  16. 2023Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studiescited by 3
  17. 2024Learning Causal Dynamics Models in Object-Oriented Environmentscited by 3
  18. 2024BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learningcited by 3
  19. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 3
  20. 2025Learning General Policies with Policy Gradient Methodscited by 3
  21. 2022Intrinsically Motivated Learning of Causal World Modelscited by 2
  22. 2024Generalizing Multi-Step Inverse Models for Representation Learning to Finite-Memory POMDPscited by 2
  23. 2024Towards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representationscited by 2
  24. 2024Dynamical-VAE-based Hindsight to Learn the Causal Dynamics of Factored-POMDPscited by 2
  25. 2026Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modelingcited by 2
  26. 2026Learning Physics-Guided Residual Dynamics for Deformable Object Simulationcited by 2
  27. 2023Beware of Instantaneous Dependence in Reinforcement Learningcited by 1
  28. 2024Reward-Relevance-Filtered Linear Offline Reinforcement Learningcited by 1
  29. 2024Why Online Reinforcement Learning is Causalcited by 1
  30. 2024On the Sample Efficiency of Abstractions and Potential-Based Reward Shaping in Reinforcement Learningcited by 1
  31. 2024On Causally Disentangled State Representation Learning for Reinforcement Learning based Recommender Systemscited by 1
  32. 2024Boosting Efficiency in Task-Agnostic Exploration through Causal Knowledgecited by 1
  33. 2025Learning Fused State Representations for Control from Multi-View Observationscited by 1
  34. 2025D3HRL: A Distributed Hierarchical Reinforcement Learning Approach Based on Causal Discovery and Spurious Correlation Detectioncited by 1
  35. 2025Situationally-Aware Dynamics Learningcited by 1
  36. 2025Learning Abstract World Models with a Group-Structured Latent Spacecited by 1
  37. 2025Efficient and Generalizable Environmental Understanding for Visual Navigationcited by 1
  38. 2025Causal Process Models: Reframing Dynamic Causal Graph Discovery as a Reinforcement Learning Problemcited by 1

50 most cited of 65 All 65 (JSON)