# Collaborative Auto-Curricula Multi-Agent Reinforcement Learning with Graph Neural Network Communication Layer for Open-ended Wildfire-Management Resource Distribution

arXiv: https://arxiv.org/abs/2204.11350 · HTML: https://mlsanity.com/paper/2204.11350/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 77 references that are in the map.
- First posted on arXiv 24 Apr 2022.

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [The NetHack Learning Environment](https://mlsanity.com/paper/2006.13760/) (2020)
- [Open-Ended Learning Leads to Generally Capable Agents](https://mlsanity.com/paper/2107.12808/) (2021)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [A review of machine learning applications in wildfire science and management](https://mlsanity.com/paper/2003.00646/) (2020)
- [Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case](https://mlsanity.com/paper/1910.07421/) (2019)
- [Next Day Wildfire Spread: A Machine Learning Data Set to Predict Wildfire Spreading from Remote-Sensing Data](https://mlsanity.com/paper/2112.02447/) (2021)
- [On the Importance of Environments in Human-Robot Coordination](https://mlsanity.com/paper/2106.10853/) (2021)
- [Learning to Play No-Press Diplomacy with Best Response Policy Iteration](https://mlsanity.com/paper/2006.04635/) (2020)
- [On the Origin of Deep Learning](https://mlsanity.com/paper/1702.07800/) (2017)
- [Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication](https://mlsanity.com/paper/1901.03887/) (2019)
- [Using Unity to Help Solve Intelligence](https://mlsanity.com/paper/2011.09294/) (2020)
- [Wildfire Monitoring in Remote Areas using Autonomous Unmanned Aerial Vehicles](https://mlsanity.com/paper/1905.00492/) (2019)
- [The Power of Communication in a Distributed Multi-Agent System](https://mlsanity.com/paper/2111.15611/) (2021)
- [Optimal algebraic Breadth-First Search for sparse graphs](https://mlsanity.com/paper/1906.03113/) (2019)

## Built on by (most cited first, 3 of 3)

- [Dynamic Collaborative Multi-Agent Reinforcement Learning Communication for Autonomous Drone Reforestation](https://mlsanity.com/paper/2211.15414/) (2022)
- [Learning to Communicate and Collaborate in a Competitive Multi-Agent Setup to Clean the Ocean from Macroplastics](https://mlsanity.com/paper/2304.05872/) (2023)
- [Wildfire Autonomous Response and Prediction Using Cellular Automata (WARP-CA)](https://mlsanity.com/paper/2407.02613/) (2024)

## Abstract (arXiv)

Most real-world domains can be formulated as multi-agent (MA) systems. Intentionality sharing agents can solve more complex tasks by collaborating, possibly in less time. True cooperative actions are beneficial for egoistic and collective reasons. However, teaching individual agents to sacrifice egoistic benefits for a better collective performance seems challenging. We build on a recently proposed Multi-Agent Reinforcement Learning (MARL) mechanism with a Graph Neural Network (GNN) communication layer. Rarely chosen communication actions were marginally beneficial. Here we propose a MARL system in which agents can help collaborators perform better while risking low individual performance. We conduct our study in the context of resource distribution for wildfire management. Communicating environmental features and partially observable fire occurrence help the agent collective to pre-emptively distribute resources. Furthermore, we introduce a procedural training environment accommodating auto-curricula and open-endedness towards better generalizability. Our MA communication proposal outperforms a Greedy Heuristic Baseline and a Single-Agent (SA) setup. We further demonstrate how auto-curricula and openendedness improves generalizability of our MA proposal.