mlsanityData to 14 Oct 2026

A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inference

arXiv 2503.17803 ↗First posted 22 Mar 2025cs.LG cs.AI cs.MAcited by 1

On the map: LLM reasoning → Causal reasoning in LLMs

Abstract (arXiv)

Causal reasoning is increasingly used in Reinforcement Learning (RL) to improve the learning process in several dimensions: efficacy of learned policies, efficiency of convergence, generalisation capabilities, safety and interpretability of behaviour. However, applications of causal reasoning to Multi-Agent RL (MARL) are still mostly unexplored. In this paper, we take the first step in investigating the opportunities and challenges of applying causal reasoning in MARL. We measure the impact of a simple form of causal augmentation in state-of-the-art MARL scenarios increasingly requiring cooperation, and with state-of-the-art MARL algorithms exploiting various degrees of collaboration between agents. Then, we discuss the positive as well as negative results achieved, giving us the chance to outline the areas where further research may help to successfully transfer causal RL to the multi-agent setting.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Causal reasoning in LLMs, one of 17 directions in LLM reasoning. Builds on 29 of its 57 references that are in the map. First posted on arXiv 22 Mar 2025.

Builds on

29 of 57 references.

References · 29

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  2. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  3. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  4. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 264
  5. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  6. 2023Causal-learn: Causal Discovery in Pythoncited by 109
  7. 2021DiBS: Differentiable Bayesian Structure Learningcited by 106
  8. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 71
  9. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 68
  10. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  11. 2023A Survey on Causal Reinforcement Learningcited by 45
  12. 2021Nonlinear Invariant Risk Minimization: A Causal Approachcited by 42
17 more
  1. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 36
  2. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 33
  3. 2022Sample-Efficient Reinforcement Learning of Partially Observable Markov Gamescited by 27
  4. 2024Multi-Agent Causal Discovery Using Large Language Modelscited by 26
  5. 2023Causal Reinforcement Learning: A Surveycited by 23
  6. 2021Causal Multi-Agent Reinforcement Learning: Review and Open Problemscited by 16
  7. 2024Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learningcited by 16
  8. 2022Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalizationcited by 12
  9. 2023Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learningcited by 10
  10. 2020Algorithms for Causal Reasoning in Probability Treescited by 8
  11. 2023MACCA: Offline Multi-agent Reinforcement Learning with Causal Credit Assignmentcited by 6
  12. 2023Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learningcited by 5
  13. 2024Challenges and Considerations in the Evaluation of Bayesian Causal Discoverycited by 5
  14. 2024Incremental effects for continuous exposurescited by 4
  15. 2024Learning Representations of Instruments for Partial Identification of Treatment Effectscited by 4
  16. 2024Discovering Minimal Reinforcement Learning Environmentscited by 3
  17. 2023Discovering Causality for Efficient Cooperation in Multi-Agent Environmentscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Causal Reflection with Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applications
  2. 2025Causal Knowledge Transfer for Multi-Agent Reinforcement Learning in Dynamic Environments
  3. 2026An Introduction to Causal Reinforcement Learning
  4. 2024Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy
  5. 2026CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning
  6. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning
  7. 2025Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies
  8. 2025A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning