mlsanityData to 14 Oct 2026

IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARL

arXiv 2306.11551 ↗First posted 20 Jun 2023cs.LG cs.MA cs.SYcited by 9

Not on the map. Why?

Abstract (arXiv)

We introduce IMP-MARL, an open-source suite of multi-agent reinforcement learning (MARL) environments for large-scale Infrastructure Management Planning (IMP), offering a platform for benchmarking the scalability of cooperative MARL methods in real-world engineering applications. In IMP, a multi-component engineering system is subject to a risk of failure due to its components' damage condition. Specifically, each agent plans inspections and repairs for a specific system component, aiming to minimise maintenance costs while cooperating to minimise system failure risk. With IMP-MARL, we release several environments including one related to offshore wind structural systems, in an effort to meet today's needs to improve management strategies to support sustainable and reliable energy systems. Supported by IMP practical engineering environments featuring up to 100 agents, we conduct a benchmark campaign, where the scalability and performance of state-of-the-art cooperative MARL methods are compared against expert-based heuristic policies. The results reveal that centralised training with decentralised execution methods scale better with the number of agents than fully centralised or decentralised RL approaches, while also outperforming expert-based heuristic policies in most IMP environments. Based on our findings, we additionally outline remaining cooperation and scalability challenges that future MARL methods should still address. Through IMP-MARL, we encourage the implementation of new environments and the further development of MARL methods.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 38 of its 73 references that are in the map. First posted on arXiv 20 Jun 2023.

Builds on

38 of 73 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  5. 2016HyperNetworkscited by 1,027
  6. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  7. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  8. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  9. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  10. 2019The StarCraft Multi-Agent Challengecited by 597
  11. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  12. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
26 more
  1. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  2. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  3. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 220
  4. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 212
  5. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
  6. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  7. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  8. 2019MAVEN: Multi-Agent Variational Explorationcited by 185
  9. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  10. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  11. 2016End-to-end Learning of Driving Models from Large-scale Video Datasetscited by 148
  12. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  13. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  14. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
  15. 2019CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenariocited by 115
  16. 2013The Optimal Reward Baseline for Gradient-Based Reinforcement Learningcited by 85
  17. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
  18. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  19. 2018Managing engineering systems with large state and action spaces through deep reinforcement learningcited by 32
  20. 2020Flatland-RL : Multi-Agent Reinforcement Learning on Trainscited by 32
  21. 2020Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraintscited by 18
  22. 2020Optimal Inspection and Maintenance Planning for Deteriorating Structural Components through Dynamic Bayesian Networks and Markov Decision Processescited by 13
  23. 2022Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learningcited by 11
  24. 2018Deep Quality-Value (DQV) Learningcited by 5
  25. 2020QVMix and QVMix-Max: Extending the Deep Quality-Value Family of Algorithms to Cooperative Multi-Agent Reinforcement Learningcited by 5
  26. 2021Optimal adaptive inspection and maintenance planning for deteriorating structural systemscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 1.

9 papers.

  1. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 21
  2. 2025Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learningcited by 7
  3. 2024MOMAland: A Set of Benchmarks for Multi-Objective Multi-Agent Reinforcement Learningcited by 6
  4. 2024Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure managementcited by 5
  5. 2024Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learningcited by 5
  6. 2025Deep Reinforcement Learning for Job Scheduling and Resource Management in Cloud Computing: An Algorithm-Level Reviewcited by 4
  7. 2025Unifying Agent Interaction and World Information for Multi-agent Coordinationcited by 2
  8. 2024Upside-Down Reinforcement Learning for More Interpretable Optimal Controlcited by 0
  9. 2026Graph-Based Inference and Topology-Aware Multi-Agent Reinforcement Learning for Large-Scale Railway Network Managementcited by 0