# Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes

arXiv: https://arxiv.org/abs/1709.06656 · HTML: https://mlsanity.com/paper/1709.06656/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2017.

## Built on by (most cited first, 21 of 21)

- [ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination](https://mlsanity.com/paper/2310.05208/) (2023)
- [Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.10113/) (2022)
- [Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approach](https://mlsanity.com/paper/2105.08268/) (2021)
- [Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL](https://mlsanity.com/paper/2209.09845/) (2022)
- [Optimizing delegation between human and AI collaborative agents](https://mlsanity.com/paper/2309.14718/) (2023)
- [Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2105.00376/) (2021)
- [Single Agent Robust Deep Reinforcement Learning for Bus Fleet Control](https://mlsanity.com/paper/2508.20784/) (2025)
- [Optimizing Delegation in Collaborative Human-AI Hybrid Teams](https://mlsanity.com/paper/2402.05605/) (2024)
- [QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers](https://mlsanity.com/paper/2410.06170/) (2024)
- [Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2209.10003/) (2022)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Survey](https://mlsanity.com/paper/2508.20315/) (2025)
- [Optimizing Risk-averse Human-AI Hybrid Teams](https://mlsanity.com/paper/2403.08386/) (2024)
- [Transferable Graphical MARL for Real-Time Estimation in Dynamic Wireless Networks](https://mlsanity.com/paper/2404.03227/) (2024)
- [H-TD2: Hybrid Temporal Difference Learning for Adaptive Urban Taxi Dispatch](https://mlsanity.com/paper/2105.02138/) (2021)
- [Multi-Agent Environments for Vehicle Routing Problems](https://mlsanity.com/paper/2411.14411/) (2024)
- [Co-TAP: Three-Layer Agent Interaction Protocol Technical Report](https://mlsanity.com/paper/2510.08263/) (2025)
- [Asynchronous Risk-Aware Multi-Agent Packet Routing for Ultra-Dense LEO Satellite Networks](https://mlsanity.com/paper/2510.27506/) (2025)
- [Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks](https://mlsanity.com/paper/2601.12662/) (2026)
- [When Does Hierarchy Help? Benchmarking Agent Coordination in Event-Driven Industrial Scheduling](https://mlsanity.com/paper/2605.13172/) (2026)
- [OpenHail: An Event-Driven Gymnasium Environment for Electric Ride-Hailing Fleet Control](https://mlsanity.com/paper/2609.30628/) (2026)

## Abstract (arXiv)

The incorporation of macro-actions (temporally extended actions) into multi-agent decision problems has the potential to address the curse of dimensionality associated with such decision problems. Since macro-actions last for stochastic durations, multiple agents executing decentralized policies in cooperative environments must act asynchronously. We present an algorithm that modifies generalized advantage estimation for temporally extended actions, allowing a state-of-the-art policy optimization algorithm to optimize policies in Dec-POMDPs in which agents act asynchronously. We show that our algorithm is capable of learning optimal policies in two cooperative domains, one involving real-time bus holding control and one involving wildfire fighting with unmanned aircraft. Our algorithm works by framing problems as "event-driven decision processes," which are scenarios in which the sequence and timing of actions and events are random and governed by an underlying stochastic process. In addition to optimizing policies with continuous state and action spaces, our algorithm also facilitates the use of event-driven simulators, which do not require time to be discretized into time-steps. We demonstrate the benefit of using event-driven simulation in the context of multiple agents taking asynchronous actions. We show that fixed time-step simulation risks obfuscating the sequence in which closely separated events occur, adversely affecting the policies learned. In addition, we show that arbitrarily shrinking the time-step scales poorly with the number of agents.