mlsanityData to 14 Oct 2026

Causal Knowledge Transfer for Multi-Agent Reinforcement Learning in Dynamic Environments

arXiv 2507.13846 ↗First posted 18 Jul 2025cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

[Context] Multi-agent reinforcement learning (MARL) has achieved notable success in environments where agents must learn coordinated behaviors. However, transferring knowledge across agents remains challenging in non-stationary environments with changing goals. [Problem] Traditional knowledge transfer methods in MARL struggle to generalize, and agents often require costly retraining to adapt. [Approach] This paper introduces a causal knowledge transfer framework that enables RL agents to learn and share compact causal representations of paths within a non-stationary environment. As the environment changes (new obstacles), agents' collisions require adaptive recovery strategies. We model each collision as a causal intervention instantiated as a sequence of recovery actions (a macro) whose effect corresponds to a causal knowledge of how to circumvent the obstacle while increasing the chances of achieving the agent's goal (maximizing cumulative reward). This recovery action macro is transferred online from a second agent and is applied in a zero-shot fashion, i.e., without retraining, just by querying a lookup model with local context information (collisions). [Results] Our findings reveal two key insights: (1) agents with heterogeneous goals were able to bridge about half of the gap between random exploration and a fully retrained policy when adapting to new environments, and (2) the impact of causal knowledge transfer depends on the interplay between environment complexity and agents' heterogeneous goals.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 20 of its 46 references that are in the map. First posted on arXiv 18 Jul 2025.

Builds on

20 of 46 references.

References · 20

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  3. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  4. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  5. 2020A Survey on Negative Transfercited by 122
  6. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  7. 2019Estimating Treatment Effects with Causal Forests: An Applicationcited by 58
  8. 2023A Survey on Causal Reinforcement Learningcited by 45
  9. 2018Multi-Agent Common Knowledge Reinforcement Learningcited by 31
  10. 2022Factored Adaptation for Non-Stationary Reinforcement Learningcited by 30
  11. 2023Causal Reinforcement Learning: A Surveycited by 23
  12. 2017Transfer Learning for Improving Model Predictions in Highly Configurable Softwarecited by 12
8 more
  1. 2021Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignmentcited by 12
  2. 2022Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalizationcited by 12
  3. 2020An Efficient Transfer Learning Framework for Multiagent Reinforcement Learningcited by 11
  4. 2021Multi-Agent Transfer Learning in Reinforcement Learning-Based Ride-Sharing Systemscited by 6
  5. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  6. 2021Evaluating Generalization and Transfer Capacity of Multi-Agent Reinforcement Learning Across Variable Number of Agentscited by 3
  7. 2024Learning Recovery Strategies for Dynamic Self-healing in Reactive Systemscited by 1
  8. 2024Enabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025AbideGym: Turning Static RL Worlds into Adaptive Challengescited by 1
  2. 2026Towards Neuro-symbolic Causal Rule Synthesis, Verification, and Evaluation Grounded in Legal and Safety Principlescited by 1
  3. 2026CrossMaps: Confidence-Aware Open-Vocabulary Semantic Mapping for Rover Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inference
  2. 2025Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination
  3. 2026CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning
  4. 2025Decentralizing Multi-Agent Reinforcement Learning with Temporal Causal Information
  5. 2024Why Online Reinforcement Learning is Causal
  6. 2023Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities
  7. 2024Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy
  8. 2026CRRL: A Causality-Based Reinforcement Learning Framework for Autonomous System Recovery