# Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.09539 · HTML: https://mlsanity.com/paper/2312.09539/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 18 of 18)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [MAVEN: Multi-Agent Variational Exploration](https://mlsanity.com/paper/1910.07483/) (2019)
- [Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning](https://mlsanity.com/paper/1509.08731/) (2015)
- [Learning Independent Causal Mechanisms](https://mlsanity.com/paper/1712.00961/) (2017)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [Causal Influence Detection for Improving Efficiency in Reinforcement Learning](https://mlsanity.com/paper/2106.03443/) (2021)
- [Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1901.09207/) (2019)
- [PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration](https://mlsanity.com/paper/2203.08553/) (2022)
- [Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies](https://mlsanity.com/paper/2103.16772/) (2021)
- [Relevant sparse codes with variational information bottleneck](https://mlsanity.com/paper/1605.07332/) (2016)
- [Multi-Agent Interactions Modeling with Correlated Policies](https://mlsanity.com/paper/2001.03415/) (2020)
- [Signal Instructed Coordination in Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1909.04224/) (2019)

## Built on by (most cited first, 10 of 10)

- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [CausalPlan: Empowering Efficient LLM Multi-Agent Collaboration Through Causality-Driven Planning](https://mlsanity.com/paper/2508.13721/) (2025)
- [Nonparametric Bellman Mappings for Value Iteration in Distributed Reinforcement Learning](https://mlsanity.com/paper/2503.16192/) (2025)
- [A Roadmap Towards Improving Multi-Agent Reinforcement Learning With Causal Discovery And Inference](https://mlsanity.com/paper/2503.17803/) (2025)
- [Hierarchical Adversarially-Resilient Multi-Agent Reinforcement Learning for Cyber-Physical Systems Security](https://mlsanity.com/paper/2506.22445/) (2025)
- [Explaining Strategic Decisions in Multi-Agent Reinforcement Learning for Aerial Combat Tactics](https://mlsanity.com/paper/2505.11311/) (2025)
- [Causal-Aware Intelligent QoE Optimization for VR Interaction with Adaptive Keyframe Extraction](https://mlsanity.com/paper/2506.19890/) (2025)
- [Understanding Action Effects through Instrumental Empowerment in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.15652/) (2025)
- [CausalGDP: Causality-Guided Diffusion Policies for Reinforcement Learning](https://mlsanity.com/paper/2602.09207/) (2026)
- [MAGIC: Multi-Step Advantage-Gated Causal Influence for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2605.01805/) (2026)

## Abstract (arXiv)

Learning to collaborate has witnessed significant progress in multi-agent reinforcement learning (MARL). However, promoting coordination among agents and enhancing exploration capabilities remain challenges. In multi-agent environments, interactions between agents are limited in specific situations. Effective collaboration between agents thus requires a nuanced understanding of when and how agents' actions influence others. To this end, in this paper, we propose a novel MARL algorithm named Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning (SCIC), which incorporates a novel Intrinsic reward mechanism based on a new cooperation criterion measured by situation-dependent causal influence among agents. Our approach aims to detect inter-agent causal influences in specific situations based on the criterion using causal intervention and conditional mutual information. This effectively assists agents in exploring states that can positively impact other agents, thus promoting cooperation between agents. The resulting update links coordinated exploration and intrinsic reward distribution, which enhance overall collaboration and performance. Experimental results on various MARL benchmarks demonstrate the superiority of our method compared to state-of-the-art approaches.