mlsanityData to 14 Oct 2026

Causally Correct Partial Models for Reinforcement Learning

arXiv 2002.02836 ↗First posted 7 Feb 2020cs.LG cs.AI stat.MLcited by 24

Not on the map. Why?

Abstract (arXiv)

In reinforcement learning, we can learn a model of future observations and rewards, and use it to plan the agent's next actions. However, jointly modeling future observations can be computationally expensive or even intractable if the observations are high-dimensional (e.g. images). For this reason, previous works have considered partial models, which model only part of the observation. In this paper, we show that partial models can be causally incorrect: they are confounded by the observations they don't model, and can therefore lead to incorrect planning. To address this, we introduce a general family of partial models that are provably causally correct, yet remain fast because they do not need to fully model future observations.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 11; 2022: 3; 2023: 3; 2024: 5; 2025: 2.

24 papers.

Sort:

By area:

  1. 2021D'ya like DAGs? A Survey on Structure Learning and Causal Discoverycited by 211
  2. 2021Towards Causal Representation Learningcited by 209
  3. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
  4. 2021Shaking the foundations: delusions in sequence models for interaction and controlcited by 62
  5. 2021Muesli: Combining Improvements in Policy Optimizationcited by 53
  6. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  7. 2021Vector Quantized Models for Planningcited by 48
  8. 2023A Survey on Causal Reinforcement Learningcited by 45
  9. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  10. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 31
  11. 2023Causal Reinforcement Learning: A Surveycited by 23
  12. 2023Automated Cyber Defence: A Reviewcited by 21
12 more
  1. 2024Causal Action Influence Aware Counterfactual Data Augmentationcited by 12
  2. 2024Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learningcited by 11
  3. 2024Do Transformer World Models Give Better Policy Gradients?cited by 10
  4. 2022On the Generalization and Adaption Performance of Causal Modelscited by 8
  5. 2021Self-Consistent Models and Valuescited by 7
  6. 2022Learning Causal Overhypotheses through Exploration in Children and Computational Modelscited by 7
  7. 2022Deconfounding Imitation Learning with Variational Inferencecited by 5
  8. 2021A Validation Tool for Designing Reinforcement Learning Environmentscited by 0
  9. 2024REVEAL-IT: REinforcement learning with Visibility of Evolving Agent poLicy for InTerpretabilitycited by 0
  10. 2024Disentangled Representations for Causal Cognitioncited by 0
  11. 2025CAIMAN: Causal Action Influence Detection for Sample-efficient Loco-manipulationcited by 0
  12. 2025Parameter Estimation using Reinforcement Learning Causal Curiosity: Limits and Challengescited by 0