# Causal Knowledge Transfer for Multi-Agent Reinforcement Learning in Dynamic Environments

arXiv: https://arxiv.org/abs/2507.13846 · HTML: https://mlsanity.com/paper/2507.13846/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 18 Jul 2025.

## Builds on (most cited first, 20 of 20)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [A Survey on Negative Transfer](https://mlsanity.com/paper/2009.00909/) (2020)
- [Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement](https://mlsanity.com/paper/1901.10964/) (2019)
- [Estimating Treatment Effects with Causal Forests: An Application](https://mlsanity.com/paper/1902.07409/) (2019)
- [A Survey on Causal Reinforcement Learning](https://mlsanity.com/paper/2302.05209/) (2023)
- [Multi-Agent Common Knowledge Reinforcement Learning](https://mlsanity.com/paper/1810.11702/) (2018)
- [Factored Adaptation for Non-Stationary Reinforcement Learning](https://mlsanity.com/paper/2203.16582/) (2022)
- [Causal Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.01452/) (2023)
- [Transfer Learning for Improving Model Predictions in Highly Configurable Software](https://mlsanity.com/paper/1704.00234/) (2017)
- [Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignment](https://mlsanity.com/paper/2106.00517/) (2021)
- [Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization](https://mlsanity.com/paper/2202.06545/) (2022)
- [An Efficient Transfer Learning Framework for Multiagent Reinforcement Learning](https://mlsanity.com/paper/2002.08030/) (2020)
- [Multi-Agent Transfer Learning in Reinforcement Learning-Based Ride-Sharing Systems](https://mlsanity.com/paper/2112.00424/) (2021)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Evaluating Generalization and Transfer Capacity of Multi-Agent Reinforcement Learning Across Variable Number of Agents](https://mlsanity.com/paper/2111.14177/) (2021)
- [Learning Recovery Strategies for Dynamic Self-healing in Reactive Systems](https://mlsanity.com/paper/2401.12405/) (2024)
- [Enabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation](https://mlsanity.com/paper/2402.08184/) (2024)

## Built on by (most cited first, 3 of 3)

- [AbideGym: Turning Static RL Worlds into Adaptive Challenges](https://mlsanity.com/paper/2509.21234/) (2025)
- [Towards Neuro-symbolic Causal Rule Synthesis, Verification, and Evaluation Grounded in Legal and Safety Principles](https://mlsanity.com/paper/2604.28087/) (2026)
- [CrossMaps: Confidence-Aware Open-Vocabulary Semantic Mapping for Rover Navigation](https://mlsanity.com/paper/2606.16935/) (2026)

## Abstract (arXiv)

[Context] Multi-agent reinforcement learning (MARL) has achieved notable success in environments where agents must learn coordinated behaviors. However, transferring knowledge across agents remains challenging in non-stationary environments with changing goals. [Problem] Traditional knowledge transfer methods in MARL struggle to generalize, and agents often require costly retraining to adapt. [Approach] This paper introduces a causal knowledge transfer framework that enables RL agents to learn and share compact causal representations of paths within a non-stationary environment. As the environment changes (new obstacles), agents' collisions require adaptive recovery strategies. We model each collision as a causal intervention instantiated as a sequence of recovery actions (a macro) whose effect corresponds to a causal knowledge of how to circumvent the obstacle while increasing the chances of achieving the agent's goal (maximizing cumulative reward). This recovery action macro is transferred online from a second agent and is applied in a zero-shot fashion, i.e., without retraining, just by querying a lookup model with local context information (collisions). [Results] Our findings reveal two key insights: (1) agents with heterogeneous goals were able to bridge about half of the gap between random exploration and a fully retrained policy when adapting to new environments, and (2) the impact of causal knowledge transfer depends on the interplay between environment complexity and agents' heterogeneous goals.