mlsanityData to 7 Oct 2026

Causal Reasoning from Meta-reinforcement Learning

arXiv 1901.08162 ↗First posted 23 Jan 2019cs.LG cs.AI stat.MLcited by 53

Not on the map. Why?

Abstract (arXiv)

Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinforcement learning. We train a recurrent network with model-free reinforcement learning to solve a range of problems that each contain causal structure. We find that the trained agent can perform causal reasoning in novel situations in order to obtain rewards. The agent can select informative interventions, draw causal inferences from observational data, and make counterfactual predictions. Although established formal causal reasoning algorithms also exist, in this paper we show that such reasoning can arise from model-free reinforcement learning, and suggest that causal reasoning in complex settings may benefit from the more end-to-end learning-based approaches presented here. This work also offers new strategies for structured exploration in reinforcement learning, by providing agents with the ability to perform -- and interpret -- experiments.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 16; 2023: 10; 2024: 9; 2025: 6; 2026: 3.

53 papers.

Sort:

By area:

  1. 2022Using cognitive psychology to understand GPT-3cited by 389
  2. 2021Towards Causal Representation Learningcited by 207
  3. 2021Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learningcited by 53
  4. 2021Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planningcited by 43
  5. 2023A Survey on Causal Reinforcement Learningcited by 43
  6. 2023Visual cognition in multimodal large language modelscited by 38
  7. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  8. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  9. 2024The broader spectrum of in-context learningcited by 33
  10. 2022Counterfactual harmcited by 29
  11. 2024Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoningcited by 24
  12. 2021Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Modelscited by 22
38 more
  1. 2023Causal Reinforcement Learning: A Surveycited by 22
  2. 2023Passive learning of active causal strategies in agents and language modelscited by 21
  3. 2022Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machinescited by 20
  4. 2023Causal Deep Learningcited by 20
  5. 2021A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directionscited by 19
  6. 2023Causal models in string diagramscited by 14
  7. 2024PUZZLES: A Benchmark for Neural Algorithmic Reasoningcited by 12
  8. 2025Anatomy of a Machine Learning Ecosystem: 2 Million Models on Hugging Facecited by 12
  9. 2022A Meta-Reinforcement Learning Algorithm for Causal Discoverycited by 11
  10. 2022Beyond Object Recognition: A New Benchmark towards Object Concept Learningcited by 11
  11. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  12. 2024Debiased Multimodal Understanding for Human Language Sequencescited by 10
  13. 2022On the Generalization and Adaption Performance of Causal Modelscited by 8
  14. 2022Learning Causal Overhypotheses through Exploration in Children and Computational Modelscited by 7
  15. 2022Disentangling Abstraction from Statistical Pattern Matching in Human and Machine Learningcited by 6
  16. 2022Intelligent Computing: The Latest Advances, Challenges and Futurecited by 6
  17. 2023A Survey of Methods, Challenges and Perspectives in Causalitycited by 6
  18. 2022Causal Counterfactuals for Improving the Robustness of Reinforcement Learningcited by 5
  19. 2024Counterfactual Influence in Markov Decision Processescited by 4
  20. 2023A Data-Driven State Aggregation Approach for Dynamic Discrete Choice Modelscited by 3
  21. 2024Automated Discovery of Functional Actual Causes in Complex Environmentscited by 3
  22. 2021Did I do that? Blame as a means to identify controlled effects in reinforcement learningcited by 2
  23. 2023Q-Cogni: An Integrated Causal Reinforcement Learning Frameworkcited by 2
  24. 2024Systems with Switching Causal Relations: A Meta-Causal Perspectivecited by 2
  25. 2025CauSight: Learning to Supersense for Visual Causal Discoverycited by 2
  26. 2021A Meta Learning Approach to Discerning Causal Graph Structurecited by 1
  27. 2022Using Contrastive Samples for Identifying and Leveraging Possible Causal Relationships in Reinforcement Learningcited by 1
  28. 2023On First-Order Meta-Reinforcement Learning with Moreau Envelopescited by 1
  29. 2025DODO: Causal Structure Learning with Budgeted Interventionscited by 1
  30. 2025Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applicationscited by 1
  31. 2026NoisyCausal: A Benchmark for Evaluating Causal Reasoning Under Structured Noisecited by 1
  32. 2022Efficiently Disentangle Causal Representationscited by 0
  33. 2022Active learning of causal probability treescited by 0
  34. 2022On a Built-in Conflict between Deep Learning and Systematic Generalizationcited by 0
  35. 2022Causal Coupled Mechanisms: A Control Method with Cooperation and Competition for Complex Systemcited by 0
  36. 2022A Short Survey of Systematic Generalizationcited by 0
  37. 2024Causal Discovery by Kernel Deviance Measures with Heterogeneous Transformscited by 0
  38. 2025A Theoretical Analysis of Provable Compositional Generalization in Neural Networks: A Necessary and Sufficient Conditioncited by 0

50 most cited of 53 All 53 (JSON)