mlsanityData to 7 Oct 2026

Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization

arXiv 2202.06545 ↗First posted 14 Feb 2022cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

In the sequential decision making setting, an agent aims to achieve systematic generalization over a large, possibly infinite, set of environments. Such environments are modeled as discrete Markov decision processes with both states and actions represented through a feature vector. The underlying structure of the environments allows the transition dynamics to be factored into two components: one that is environment-specific and another that is shared. Consider a set of environments that share the laws of motion as an example. In this setting, the agent can take a finite amount of reward-free interactions from a subset of these environments. The agent then must be able to approximately solve any planning task defined over any environment in the original set, relying on the above interactions only. Can we design a provably efficient algorithm that achieves this ambitious goal of systematic generalization? In this paper, we give a partially positive answer to this question. First, we provide a tractable formulation of systematic generalization by employing a causal viewpoint. Then, under specific structural assumptions, we provide a simple learning algorithm that guarantees any desired planning error up to an unavoidable sub-optimality term, while showcasing a polynomial sample complexity.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 28 of its 60 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

28 of 60 references.

References · 28

  1. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  2. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
  3. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
  4. 2019Provably Efficient Exploration in Policy Optimizationcited by 203
  5. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  6. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  7. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  8. 2019Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learningcited by 114
  9. 2020Invariant Causal Prediction for Block MDPscited by 104
  10. 2020Fast active learning for pure exploration in reinforcement learningcited by 92
  11. 2020On Reward-Free Reinforcement Learning with Linear Function Approximationcited by 89
  12. 2020Adaptive Reward-Free Explorationcited by 77
16 more
  1. 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
  2. 2020Provably Efficient Reward-Agnostic Navigation with Linear Value Iterationcited by 56
  3. 2014Near-optimal Reinforcement Learning in Factored MDPscited by 41
  4. 2017Budgeted Experiment Design for Causal Structure Learningcited by 38
  5. 2021Causal Reinforcement Learning using Observational and Interventional Datacited by 38
  6. 2017Testing Conditional Independence of Discrete Distributionscited by 22
  7. 2021Efficient Local Planning with Linear Function Approximationcited by 21
  8. 2020Active Model Estimation in Markov Decision Processescited by 20
  9. 2021On Reward-Free RL with Kernel and Neural Function Approximations: Single-Agent MDP and Markov Gamecited by 20
  10. 2021Model-Invariant State Abstractions for Model-Based Reinforcement Learningcited by 18
  11. 2020Optimal Testing of Discrete Distributions with High Probabilitycited by 15
  12. 2020Towards Minimax Optimal Reinforcement Learning in Factored Markov Decision Processescited by 13
  13. 2013On the Sample Complexity of Learning Bayesian Networkscited by 9
  14. 2021On the Sample Complexity of Causal Discovery and the Value of Domain Expertisecited by 9
  15. 2020Oracle-Efficient Regret Minimization in Factored MDPs with Unknown Structurecited by 6
  16. 2020Improved Exploration in Factored Average-Reward MDPscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 4; 2025: 4; 2026: 2.

12 papers.

  1. 2022Factored Adaptation for Non-Stationary Reinforcement Learningcited by 30
  2. 2024Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learningcited by 10
  3. 2023Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learningcited by 5
  4. 2025Towards Empowerment Gain through Causal Structure Learning in Model-Based RLcited by 4
  5. 2024BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learningcited by 3
  6. 2025Causal Knowledge Transfer for Multi-Agent Reinforcement Learning in Dynamic Environmentscited by 3
  7. 2024Boosting Efficiency in Task-Agnostic Exploration through Causal Knowledgecited by 1
  8. 2025Adaptive Data Exploitation in Deep Reinforcement Learningcited by 1
  9. 2025A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inferencecited by 1
  10. 2024Disentangled Representations for Causal Cognitioncited by 0
  11. 2026Transferable Delay-Aware Reinforcement Learning via Implicit Causal Graph Modelingcited by 0
  12. 2026When Do Causal World Models Help Modular LLM Agentscited by 0