The StarCraft Multi-Agent Challenges+ : Learning of Multi-Stage Tasks and Environmental Factors without Precise Reward Functions
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose a novel benchmark called the StarCraft Multi-Agent Challenges+, where agents learn to perform multi-stage tasks and to use environmental factors without precise reward functions. The previous challenges (SMAC) recognized as a standard benchmark of Multi-Agent Reinforcement Learning are mainly concerned with ensuring that all agents cooperatively eliminate approaching adversaries only through fine manipulation with obvious reward functions. This challenge, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control. This study covers both offensive and defensive scenarios. In the offensive scenarios, agents must learn to first find opponents and then eliminate them. The defensive scenarios require agents to use topographic features. For example, agents need to position themselves behind protective structures to make it harder for enemies to attack. We investigate MARL algorithms under SMAC+ and observe that recent approaches work well in similar settings to the previous challenges, but misbehave in offensive scenarios. Additionally, we observe that an enhanced exploration approach has a positive effect on performance but is not able to completely solve all scenarios. This study proposes new directions for future research.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 26 references that are in the map. First posted on arXiv 5 Jul 2022.
Builds on
20 of 26 references.
References · 20
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
- 2017A Distributional Perspective on Reinforcement Learningcited by 743
- 2017Counterfactual Multi-Agent Policy Gradientscited by 733
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
- 2019The StarCraft Multi-Agent Challengecited by 589
- 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
- 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
- 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
- 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
- 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
8 more
- 2019Fully Parameterized Quantile Function for Distributional Reinforcement Learningcited by 87
- 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
- 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
- 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
- 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
- 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
- 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
- 2021Decomposed Soft Actor-Critic Method for Cooperative Multi-Agent Reinforcement Learningcited by 10
Built on by
Citing papers per year: 2023: 1.
1 paper.