DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Learning optimal behavior policy for each agent in multi-agent systems is an essential yet difficult problem. Despite fruitful progress in multi-agent reinforcement learning, the challenge of addressing the dynamics of whether two agents should exhibit consistent behaviors is still under-explored. In this paper, we propose a new approach that enables agents to learn whether their behaviors should be consistent with that of other agents by utilizing intrinsic rewards to learn the optimal policy for each agent. We begin by defining behavior consistency as the divergence in output actions between two agents when provided with the same observation. Subsequently, we introduce dynamic consistency intrinsic reward (DCIR) to stimulate agents to be aware of others' behaviors and determine whether to be consistent with them. Lastly, we devise a dynamic scale network (DSN) that provides learnable scale factors for the agent at every time step to dynamically ascertain whether to award consistent behavior and the magnitude of rewards. We evaluate DCIR in multiple environments including Multi-agent Particle, Google Research Football and StarCraft II Micromanagement, demonstrating its efficacy.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 52 references that are in the map. First posted on arXiv 10 Dec 2023.
Builds on
28 of 52 references.
References · 28
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2015Trust Region Policy Optimizationcited by 2,580
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
- 2017Counterfactual Multi-Agent Policy Gradientscited by 751
- 2019The StarCraft Multi-Agent Challengecited by 597
- 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
- 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
16 more
- 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
- 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
- 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
- 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
- 2015Incentivizing Exploration In Reinforcement Learning With Deep Predictive Modelscited by 154
- 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
- 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 102
- 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
- 2019Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learningcited by 30
- 2022Learning Active Camera for Multi-Object Navigationcited by 21
- 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 20
- 2019A New Framework for Multi-Agent Reinforcement Learning -- Centralized Training and Exploration with Decentralized Execution via Policy Distillationcited by 12
- 2022ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Rewardcited by 11
- 2019Stigmergic Independent Reinforcement Learning for Multi-Agent Collaborationcited by 8
- 2021Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problemcited by 5
- 2023AIIR-MIX: Multi-Agent Reinforcement Learning Meets Attention Individual Intrinsic Reward Mixing Networkcited by 1
Built on by
Not cited yet.