mlsanityData to 7 Oct 2026

Improving Global Parameter-sharing in Physically Heterogeneous Multi-agent Reinforcement Learning with Unified Action Space

arXiv 2408.07395 ↗First posted 14 Aug 2024cs.MA cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In a multi-agent system (MAS), action semantics indicates the different influences of agents' actions toward other entities, and can be used to divide agents into groups in a physically heterogeneous MAS. Previous multi-agent reinforcement learning (MARL) algorithms apply global parameter-sharing across different types of heterogeneous agents without careful discrimination of different action semantics. This common implementation decreases the cooperation and coordination between agents in complex situations. However, fully independent agent parameters dramatically increase the computational cost and training difficulty. In order to benefit from the usage of different action semantics while also maintaining a proper parameter-sharing structure, we introduce the Unified Action Space (UAS) to fulfill the requirement. The UAS is the union set of all agent actions with different semantics. All agents first calculate their unified representation in the UAS, and then generate their heterogeneous action policies using different available-action-masks. To further improve the training of extra UAS parameters, we introduce a Cross-Group Inverse (CGI) loss to predict other groups' agent policies with the trajectory information. As a universal method for solving the physically heterogeneous MARL problem, we implement the UAS adding to both value-based and policy-based MARL algorithms, and propose two practical algorithms: U-QMIX and U-MAPPO. Experimental results in the SMAC environment prove the effectiveness of both U-QMIX and U-MAPPO compared with several state-of-the-art MARL methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 50 references that are in the map. First posted on arXiv 14 Aug 2024.

Builds on

31 of 50 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  6. 2019The StarCraft Multi-Agent Challengecited by 589
  7. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  8. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  9. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  10. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  11. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  12. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 173
19 more
  1. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  2. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  3. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  4. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  5. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  6. 2024Designing Heterogeneous LLM Agents for Financial Sentiment Analysiscited by 48
  7. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
  8. 2019Action Semantics Network: Considering the Effects of Actions in Multiagent Systemscited by 19
  9. 2022UNMAS: Multi-Agent Reinforcement Learning for Unshaped Cooperative Scenarioscited by 9
  10. 2021Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RLcited by 7
  11. 2024Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaborationcited by 6
  12. 2020Multi-Agent Deep Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks with Imperfect Channelscited by 5
  13. 2019Motion control for autonomous heterogeneous multi-agent area search in uncertain conditionscited by 4
  14. 2024HeR-DRL:Heterogeneous Relational Deep Reinforcement Learning for Decentralized Multi-Robot Crowd Navigationcited by 3
  15. 2018Learning to Communicate: A Machine Learning Framework for Heterogeneous Multi-Agent Robotic Systemscited by 2
  16. 2022Multi-Agent Reinforcement Learning for Network Selection and Resource Allocation in Heterogeneous multi-RAT Networkscited by 2
  17. 2023Multi-Agent Reinforcement Learning for Power Control in Wireless Networks via Adaptive Graphscited by 2
  18. 2023Decentralized traffic management of autonomous dronescited by 2
  19. 2024Battery-Swapping Multi-Agent System for Sustained Operation of Large Planetary Fleetscited by 1

Built on by

Not cited yet.