# Multi-agent Dynamic Algorithm Configuration

arXiv: https://arxiv.org/abs/2210.06835 · HTML: https://mlsanity.com/paper/2210.06835/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 73 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 28 of 28)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://mlsanity.com/paper/1810.05587/) (2018)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [Tianshou: a Highly Modularized Deep Reinforcement Learning Library](https://mlsanity.com/paper/2107.14171/) (2021)
- [Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1906.04737/) (2019)
- [ParamILS: An Automatic Algorithm Configuration Framework](https://mlsanity.com/paper/1401.3492/) (2014)
- [NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search](https://mlsanity.com/paper/2007.10396/) (2020)
- [MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence](https://mlsanity.com/paper/1712.00600/) (2017)
- [Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks](https://mlsanity.com/paper/2110.14300/) (2021)
- [Towards a Standardised Performance Evaluation Protocol for Cooperative MARL](https://mlsanity.com/paper/2209.10485/) (2022)
- [PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization](https://mlsanity.com/paper/1701.00879/) (2017)
- [Fair Algorithms for Multi-Agent Multi-Armed Bandits](https://mlsanity.com/paper/2007.06699/) (2020)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [Deep Reinforcement Learning Based Parameter Control in Differential Evolution](https://mlsanity.com/paper/1905.08006/) (2019)
- [DACBench: A Benchmark Library for Dynamic Algorithm Configuration](https://mlsanity.com/paper/2105.08541/) (2021)
- [Learning adaptive differential evolution algorithm from optimization experiences by policy gradient](https://mlsanity.com/paper/2102.03572/) (2021)
- [Learning Heuristic Selection with Dynamic Algorithm Configuration](https://mlsanity.com/paper/2006.08246/) (2020)
- [Heterogeneous Multi-Agent Reinforcement Learning for Unknown Environment Mapping](https://mlsanity.com/paper/2010.02663/) (2020)
- [Theory-inspired Parameter Control Benchmarks for Dynamic Algorithm Configuration](https://mlsanity.com/paper/2202.03259/) (2022)
- [Decomposition Multi-Objective Evolutionary Optimization: From State-of-the-Art to Future Opportunities](https://mlsanity.com/paper/2108.09588/) (2021)
- [What Weights Work for You? Adapting Weights for Any Pareto Front Shape in Decomposition-based Evolutionary Multi-Objective Optimisation](https://mlsanity.com/paper/1709.02679/) (2017)

## Built on by (most cited first, 30 of 36)

- [Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms](https://mlsanity.com/paper/2401.11963/) (2024)
- [Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution](https://mlsanity.com/paper/2403.02131/) (2024)
- [Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization](https://mlsanity.com/paper/2411.00625/) (2024)
- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers](https://mlsanity.com/paper/2305.05909/) (2023)
- [Reinforced In-Context Black-Box Optimization](https://mlsanity.com/paper/2402.17423/) (2024)
- [Multi-agent Continual Coordination via Progressive Task Contextualization](https://mlsanity.com/paper/2305.13937/) (2023)
- [Neural Exploratory Landscape Analysis for Meta-Black-Box-Optimization](https://mlsanity.com/paper/2408.10672/) (2024)
- [MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization](https://mlsanity.com/paper/2505.17745/) (2025)
- [Self-Motivated Multi-Agent Exploration](https://mlsanity.com/paper/2301.02083/) (2023)
- [Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation](https://mlsanity.com/paper/2305.05116/) (2023)
- [Open Ad Hoc Teamwork with Cooperative Game Theory](https://mlsanity.com/paper/2402.15259/) (2024)
- [Using Automated Algorithm Configuration for Parameter Control](https://mlsanity.com/paper/2302.12334/) (2023)
- [DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization](https://mlsanity.com/paper/2505.17866/) (2025)
- [BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placement](https://mlsanity.com/paper/2510.23472/) (2025)
- [Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimization](https://mlsanity.com/paper/2504.17578/) (2025)
- [Robust Multi-agent Communication via Multi-view Message Certification](https://mlsanity.com/paper/2305.13936/) (2023)
- [Sequential Multi-Agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2510.23535/) (2025)
- [Efficient Communication via Self-supervised Information Aggregation for Online and Offline Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2302.09605/) (2023)
- [Reliable and Efficient Multi-Agent Coordination via Graph Neural Network Variational Autoencoders](https://mlsanity.com/paper/2503.02954/) (2025)
- [Large Language Model-Driven Surrogate-Assisted Evolutionary Algorithm for Expensive Optimization](https://mlsanity.com/paper/2507.02892/) (2025)
- [CuDA2: An approach for Incorporating Traitor Agents into Cooperative Multi-Agent Systems](https://mlsanity.com/paper/2406.17425/) (2024)
- [Adaptive and Robust DBSCAN with Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2505.04339/) (2025)
- [Multi-parameter Control for the (1+(λ,λ))-GA on OneMax via Deep Reinforcement Learning](https://mlsanity.com/paper/2505.12982/) (2025)
- [Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review](https://mlsanity.com/paper/2507.10142/) (2025)
- [Learning Where, What and How to Transfer: A Multi-Role Reinforcement Learning Approach for Evolutionary Multitasking](https://mlsanity.com/paper/2511.15199/) (2025)
- [Meta-Black-Box Optimization with Ensemble Surrogate Modeling for Robustness-Accuracy Trade-off within SAEA](https://mlsanity.com/paper/2606.00862/) (2026)
- [Hybrid Algorithm Selection and Hyperparameter Tuning on Distributed Machine Learning Resources: A Hierarchical Agent-based Approach](https://mlsanity.com/paper/2309.06604/) (2023)
- [Multi-Agent Inverse Q-Learning from Demonstrations](https://mlsanity.com/paper/2503.04679/) (2025)
- [Interpreting Graph Inference with Skyline Explanations](https://mlsanity.com/paper/2505.07635/) (2025)

## Abstract (arXiv)

Automated algorithm configuration relieves users from tedious, trial-and-error tuning tasks. A popular algorithm configuration tuning paradigm is dynamic algorithm configuration (DAC), in which an agent learns dynamic configuration policies across instances by reinforcement learning (RL). However, in many complex algorithms, there may exist different types of configuration hyperparameters, and such heterogeneity may bring difficulties for classic DAC which uses a single-agent RL policy. In this paper, we aim to address this issue and propose multi-agent DAC (MA-DAC), with one agent working for one type of configuration hyperparameter. MA-DAC formulates the dynamic configuration of a complex algorithm with multiple types of hyperparameters as a contextual multi-agent Markov decision process and solves it by a cooperative multi-agent RL (MARL) algorithm. To instantiate, we apply MA-DAC to a well-known optimization algorithm for multi-objective optimization problems. Experimental results show the effectiveness of MA-DAC in not only achieving superior performance compared with other configuration tuning approaches based on heuristic rules, multi-armed bandits, and single-agent RL, but also being capable of generalizing to different problem classes. Furthermore, we release the environments in this paper as a benchmark for testing MARL algorithms, with the hope of facilitating the application of MARL.