Data to 14 Oct 2026

Invariant Causal Prediction for Block MDPs

arXiv 2003.06016 ↗First posted 12 Mar 2020cs.LG cs.AI stat.MLcited by 105

Not on the map. Why?

Cited by 21 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges. In this paper, we consider the problem of learning abstractions that generalize in block MDPs, families of environments with a shared latent state space and dynamics structure over that latent space, but varying observations. We leverage tools from causal inference to propose a method of invariant prediction to learn model-irrelevance state abstractions (MISA) that generalize to novel observations in the multi-environment setting. We prove that for certain classes of environments, this approach outputs with high probability a state abstraction corresponding to the causal feature set with respect to the return. We further provide more general bounds on model error and generalization error in the multi-environment setting, in the process showing a connection between causal variable selection and the state abstraction framework for MDPs. We give empirical evidence that our methods work in both linear and nonlinear settings, attaining improved generalization over single- and multi-task baselines.

Key facts. Cited by 105 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 22; 2023: 29; 2024: 21; 2025: 10; 2026: 6.

105 papers.

Sort:

By area:

2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 65
2021Learning Task Informed Abstractionscited by 58
2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 52
2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 48
2023A Survey on Causal Reinforcement Learningcited by 45
2021Nonlinear Invariant Risk Minimization: A Causal Approachcited by 42
2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 36
2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 34

38 more

2024The Essential Role of Causality in Foundation World Models for Embodied AIcited by 32
2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 31
2021Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fixcited by 27
2023Conditional Mutual Information for Disentangled Representations in Reinforcement Learningcited by 27
2023The Generalization Gap in Offline Reinforcement Learningcited by 26
2022Hyperbolic Deep Reinforcement Learningcited by 25
2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 25
2023Provable Reward-Agnostic Preference-Based Reinforcement Learningcited by 25
2021Learning Representations for Pixel-based Control: What Matters and Why?cited by 24
2023Causal Reinforcement Learning: A Surveycited by 23
2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
2023Explainable Reinforcement Learning via a Causal World Modelcited by 22
2024Learning Latent Dynamic Robust Representations for World Modelscited by 21
2022Improving Policy Optimization with Generalist-Specialist Learningcited by 20
2021Domain Adversarial Reinforcement Learningcited by 19
2022Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learningcited by 19
2021Model-Invariant State Abstractions for Model-Based Reinforcement Learningcited by 18
2022Transfer RL across Observation Feature Spaces via Model-Based Regularizationcited by 18
2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 18
2021Invariant Policy Learning: A Causal Perspectivecited by 17
2023Environment Invariant Linear Least Squarescited by 17
2021Optimization-based Causal Estimation from Heterogenous Environmentscited by 16
2022Variational Causal Dynamics: Discovering Modular World Models from Interventionscited by 16
2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
2022A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learningcited by 15
2023Invariant Causal Imitation Learning for Generalizable Policiescited by 15
2022Learning Dynamics and Generalization in Reinforcement Learningcited by 14
2021Learning Domain Invariant Representations in Goal-conditioned Block MDPscited by 12
2022Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalizationcited by 12
2021Reimagining an autonomous vehiclecited by 11
2023ReCoRe: Regularized Contrastive Representation Learning of World Modelcited by 11
2024Building Minimal and Reusable Causal State Abstractions for Reinforcement Learningcited by 11
2024Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learningcited by 11
2024Entity-based Reinforcement Learning for Autonomous Cyber Defencecited by 11
2023Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RLcited by 10
2025Four Principles for Physically Interpretable World Modelscited by 10
2021Training a Resilient Q-Network against Observational Interferencecited by 9

50 most cited of 105 All 105 (JSON)