Trajectory-Class-Aware Multi-Agent Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
In the context of multi-agent reinforcement learning, generalization is a challenge to solve various tasks that may require different joint policies or coordination without relying on policies specialized for each task. We refer to this type of problem as a multi-task, and we train agents to be versatile in this multi-task setting through a single training process. To address this challenge, we introduce TRajectory-class-Aware Multi-Agent reinforcement learning (TRAMA). In TRAMA, agents recognize a task type by identifying the class of trajectories they are experiencing through partial observations, and the agents use this trajectory awareness or prediction as additional information for action policy. To this end, we introduce three primary objectives in TRAMA: (a) constructing a quantized latent space to generate trajectory embeddings that reflect key similarities among them; (b) conducting trajectory clustering using these trajectory embeddings; and (c) building a trajectory-class-aware policy. Specifically for (c), we introduce a trajectory-class predictor that performs agent-wise predictions on the trajectory class; and we design a trajectory-class representation model for each trajectory class. Each agent takes actions based on this trajectory-class representation along with its partial observation for task-aware execution. The proposed method is evaluated on various tasks, including multi-task problems built upon StarCraft II. Empirical results show further performance improvements over state-of-the-art baselines.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 3 Mar 2025.
Builds on
37 of 50 references.
References · 37
- 2017Neural Discrete Representation Learningcited by 4,650
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2018DeepMind Control Suitecited by 935
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
- 2019The StarCraft Multi-Agent Challengecited by 589
- 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
- 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
25 more
- 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
- 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
- 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
- 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
- 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
- 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
- 2016Opponent Modeling in Deep Reinforcement Learningcited by 94
- 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 85
- 2019Discretizing Continuous Action Space for On-Policy Optimizationcited by 65
- 2018Modeling Others using Oneself in Multi-Agent Reinforcement Learningcited by 64
- 2020Agent Modelling under Partial Observability for Deep Reinforcement Learningcited by 56
- 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
- 2018Learning Policy Representations in Multiagent Systemscited by 48
- 2019Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discoverycited by 42
- 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
- 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
- 2013Experiments with Random Projectioncited by 27
- 2021Model-Based Opponent Modelingcited by 21
- 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 19
- 2020Variational Autoencoders for Opponent Modeling in Multi-Agent Systemscited by 18
- 2022LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learningcited by 15
- 2019A Structured Prediction Approach for Generalization in Cooperative Multi-Agent Reinforcement Learningcited by 10
- 2023Neural Episodic Control with State Abstractioncited by 7
- 2024LAGMA: LAtent Goal-guided Multi-Agent Reinforcement Learningcited by 5
- 2024Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learningcited by 4
Built on by
Citing papers per year: 2025: 2.
2 papers.