mlsanityData to 14 Oct 2026

Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learning

arXiv 2505.08630 ↗First posted 13 May 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Training cooperative agents in sparse-reward scenarios poses significant challenges for multi-agent reinforcement learning (MARL). Without clear feedback on actions at each step in sparse-reward setting, previous methods struggle with precise credit assignment among agents and effective exploration. In this paper, we introduce a novel method to deal with both credit assignment and exploration problems in reward-sparse domains. Accordingly, we propose an algorithm that calculates the Influence Scope of Agents (ISA) on states by taking specific value of the dimensions/attributes of states that can be influenced by individual agents. The mutual dependence between agents' actions and state attributes are then used to calculate the credit assignment and to delimit the exploration space for each individual agent. We then evaluate ISA in a variety of sparse-reward multi-agent scenarios. The results show that our method significantly outperforms the state-of-art baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

27 of 46 references.

References · 27

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  2. 2017Hindsight Experience Replaycited by 1,136
  3. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  4. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  5. 2019The StarCraft Multi-Agent Challengecited by 597
  6. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  7. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  8. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  9. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
  10. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 137
  11. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  12. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 98
15 more
  1. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  2. 2019Influence-Based Multi-Agent Explorationcited by 75
  3. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  4. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 59
  5. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  6. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  7. 2018CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learningcited by 31
  8. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  9. 2022ALMA: Hierarchical Learning for Composite Multi-Agent Taskscited by 18
  10. 2014Changing the Environment Based on Empowerment as Intrinsic Motivationcited by 10
  11. 2022Agent-Temporal Attention for Reward Redistribution in Episodic Multi-Agent Reinforcement Learningcited by 9
  12. 2023FoX: Formation-aware exploration in multi-agent reinforcement learningcited by 9
  13. 2022Agent-Time Attention for Sparse Rewards Multi-Agent Reinforcement Learningcited by 6
  14. 2023Multi-Target Pursuit by a Decentralized Heterogeneous UAV Swarm using Deep Multi-Agent Reinforcement Learningcited by 5
  15. 2020An Empowerment-based Solution to Robotic Manipulation Tasks with Sparse Rewardscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Focusing Influence Mechanism for Multi-Agent Reinforcement Learning
  2. 2026Explicit Credit Assignment through Local Rewards and Dependence Graphs in Multi-Agent Reinforcement Learning
  3. 2024Assigning Credit with Partial Reward Decoupling in Multi-Agent Proximal Policy Optimization
  4. 2025MIR: Efficient Exploration in Episodic Multi-Agent Reinforcement Learning via Mutual Intrinsic Reward
  5. 2024Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement Learning
  6. 2024Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning
  7. 2024MESA: Cooperative Meta-Exploration in Multi-Agent Learning through Exploiting State-Action Space Structure
  8. 2025Multi-level Advantage Credit Assignment for Cooperative Multi-Agent Reinforcement Learning