mlsanityData to 14 Oct 2026

Agent-Time Attention for Sparse Rewards Multi-Agent Reinforcement Learning

arXiv 2210.17540 ↗First posted 31 Oct 2022cs.LG cs.MAcited by 6

Not on the map. Why?

Abstract (arXiv)

Sparse and delayed rewards pose a challenge to single agent reinforcement learning. This challenge is amplified in multi-agent reinforcement learning (MARL) where credit assignment of these rewards needs to happen not only across time, but also across agents. We propose Agent-Time Attention (ATA), a neural network model with auxiliary losses for redistributing sparse and delayed rewards in collaborative MARL. We provide a simple example that demonstrates how providing agents with their own local redistributed rewards and shared global redistributed rewards motivate different policies. We extend several MiniGrid environments, specifically MultiRoom and DoorKey, to the multi-agent sparse delayed rewards setting. We demonstrate that ATA outperforms various baselines on many instances of these environments. Source code of the experiments is available at https://github.com/jshe/agent-time-attention.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

23 of 44 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  4. 2021Transformers in Vision: A Surveycited by 1,013
  5. 2020Efficient Transformers: A Surveycited by 790
  6. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  8. 2019The StarCraft Multi-Agent Challengecited by 597
  9. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  10. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  11. 2019MAVEN: Multi-Agent Variational Explorationcited by 185
  12. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
11 more
  1. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  2. 2018Graph Convolutional Reinforcement Learningcited by 152
  3. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  4. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 106
  5. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  6. 2020Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learningcited by 60
  7. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  8. 2018Deep Reinforcement Learning for Swarm Systemscited by 45
  9. 2020Multi-Agent Collaboration via Reward Attribution Decompositioncited by 17
  10. 2022Agent-Temporal Attention for Reward Redistribution in Episodic Multi-Agent Reinforcement Learningcited by 9
  11. 2021Coordinated Reinforcement Learning for Optimizing Mobile Networkscited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 3.

6 papers.

  1. 2023A Review of Cooperation in Multi-agent Learningcited by 29
  2. 2024Beyond Joint Demonstrations: Personalized Expert Guidance for Efficient Multi-Agent Reinforcement Learningcited by 5
  3. 2025Preference-Guided Learning for Sparse-Reward Multi-Agent Reinforcement Learningcited by 2
  4. 2024Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement Learningcited by 1
  5. 2025The challenge of hidden gifts in multi-agent reinforcement learningcited by 1
  6. 2025Credit Assignment and Efficient Exploration based on Influence Scope in Multi-agent Reinforcement Learningcited by 0