mlsanityData to 7 Oct 2026

Explainable Reinforcement Learning via a Causal World Model

arXiv 2305.02749 ↗First posted 4 May 2023cs.LGcited by 22

On the map: World models → World models for language agents

Abstract (arXiv)

Generating explanations for reinforcement learning (RL) is challenging as actions may produce long-term effects on the future. In this paper, we develop a novel framework for explainable RL by learning a causal world model without prior knowledge of the causal structure of the environment. The model captures the influence of actions, allowing us to interpret the long-term effects of actions through causal chains, which present how actions influence environmental variables and finally lead to rewards. Different from most explanatory models which suffer from low accuracy, our model remains accurate while improving explainability, making it applicable in model-based learning. As a result, we demonstrate that our causal model can serve as the bridge between explainability and learning.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 29 of its 56 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

29 of 56 references.

References · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2016OpenAI Gymcited by 1,684
  4. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  6. 2019The StarCraft Multi-Agent Challengecited by 589
  7. 2019Model-Based Reinforcement Learning for Ataricited by 495
  8. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  9. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  10. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  11. 2020Explainable Reinforcement Learning: A Surveycited by 139
  12. 2020Invariant Causal Prediction for Block MDPscited by 104
17 more
  1. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 70
  2. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 64
  3. 2019Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learningcited by 60
  4. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  5. 2021Shapley Explanation Networkscited by 41
  6. 2020Provably Efficient Causal Reinforcement Learning with Confounded Observational Datacited by 40
  7. 2018Learning Finite State Representations of Recurrent Policy Networkscited by 37
  8. 2018Deconfounding Reinforcement Learning in Observational Settingscited by 36
  9. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
  10. 2019Causal Induction from Visual Observations for Goal Directed Taskscited by 31
  11. 2018Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learningcited by 24
  12. 2019Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based roboticscited by 21
  13. 2020What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomescited by 20
  14. 2018Fast Conditional Independence Test for Vector Variables with Large Sample Sizescited by 16
  15. 2019Free-Lunch Saliency via Attention in Atari Agentscited by 12
  16. 2018Autonomous Self-Explanation of Behavior for Interactive Reinforcement Learning Agentscited by 10
  17. 2021Training a Resilient Q-Network against Observational Interferencecited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 7.

22 papers.

Sort:

By area:

  1. 2024Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6Gcited by 30
  2. 2024Making Large Language Models into World Models with Precondition and Effect Knowledgecited by 21
  3. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  4. 2025Continual Learning, Not Training: Online Adaptation For Agentscited by 4
  5. 2024Learning Causal Dynamics Models in Object-Oriented Environmentscited by 3
  6. 2025Learning Local Causal World Models with State Space Models and Attentioncited by 2
  7. 2024Boosting Efficiency in Task-Agnostic Exploration through Causal Knowledgecited by 1
  8. 2025Towards Causal Model-Based Policy Optimizationcited by 1
  9. 2025Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applicationscited by 1
  10. 2026ProPlay: Procedural World Models for Self-Evolving LLM Agentscited by 1
  11. 2026Self-Evolving Cognitive Framework via Causal World Modeling for Embodied Scientific Intelligencecited by 1
  12. 2024Transformers and Slot Encoding for Sample Efficient Physical World Modellingcited by 0
10 more
  1. 2025Compositional Concept-Based Neuron-Level Interpretability for Deep Reinforcement Learningcited by 0
  2. 2025Generating Causal Explanations of Vehicular Agent Behavioural Interactions with Learnt Reward Profilescited by 0
  3. 2025Hierarchical Reinforcement Learning with Targeted Causal Interventionscited by 0
  4. 2025Understanding Action Effects through Instrumental Empowerment in Multi-Agent Reinforcement Learningcited by 0
  5. 2025Causal Model-Based Reinforcement Learning for Sample-Efficient IoT Channel Accesscited by 0
  6. 2026CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learningcited by 0
  7. 2026Continual learning and refinement of causal models through dynamic predicate inventioncited by 0
  8. 2026Transferable Delay-Aware Reinforcement Learning via Implicit Causal Graph Modelingcited by 0
  9. 2026Sample-Efficient Learning of Probabilistic Causes for Reachability in Markov Decision Processes with Probabilistic Guaranteescited by 0
  10. 2026When Do Causal World Models Help Modular LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Explainable Reinforcement Learning Agents Using World Models
  2. 2025Better Decisions through the Right Causal World Model
  3. 2022Offline Reinforcement Learning with Causal Structured World Models
  4. 2026An Introduction to Causal Reinforcement Learning
  5. 2024Why Online Reinforcement Learning is Causal
  6. 2026Learning Implicit Causal World Models from Multi-Agent Demonstrations
  7. 2023A Closer Look at Reward Decomposition for High-Level Robotic Explanations
  8. 2025Causal Reflection with Language Models