mlsanityData to 7 Oct 2026

Multi-Agent Policy Transfer via Task Relationship Modeling

arXiv 2203.04482 ↗First posted 9 Mar 2022cs.AI cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Team adaptation to new cooperative tasks is a hallmark of human intelligence, which has yet to be fully realized in learning agents. Previous work on multi-agent transfer learning accommodate teams of different sizes, heavily relying on the generalization ability of neural networks for adapting to unseen tasks. We believe that the relationship among tasks provides the key information for policy adaptation. In this paper, we try to discover and exploit common structures among tasks for more efficient transfer, and propose to learn effect-based task representations as a common space of tasks, using an alternatively fixed training scheme. We demonstrate that the task representation can capture the relationship among tasks, and can generalize to unseen tasks. As a result, the proposed method can help transfer learned cooperation knowledge to new tasks after training on a few source tasks. We also find that fine-tuning the transferred policies help solve tasks that are hard to learn from scratch.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 50 of its 72 references that are in the map. First posted on arXiv 9 Mar 2022.

Builds on

50 of 72 references.

References · 50

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  4. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  5. 2019The StarCraft Multi-Agent Challengecited by 589
  6. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  7. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  8. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  9. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  10. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  11. 2016Learning to reinforcement learncited by 369
  12. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
38 more
  1. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 259
  2. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 170
  3. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  4. 2018Graph Convolutional Reinforcement Learningcited by 148
  5. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  6. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  7. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  8. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 114
  9. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
  10. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 97
  11. 2020Learning Latent Representations to Influence Multi-Agent Interactioncited by 92
  12. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  13. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 85
  14. 2019Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularitycited by 68
  15. 2020My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Controlcited by 68
  16. 2018Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddingscited by 52
  17. 2018Learning Policy Representations in Multiagent Systemscited by 48
  18. 2020Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learningcited by 47
  19. 2017Divide-and-Conquer Reinforcement Learningcited by 46
  20. 2018Learning to Teach in Cooperative Multiagent Reinforcement Learningcited by 45
  21. 2019Learning Transferable Cooperative Behavior in Multi-Agent Teamscited by 40
  22. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  23. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  24. 2019From Few to More: Large-scale Dynamic Multiagent Curriculum Learningcited by 36
  25. 2019Regularizing Deep Multi-Task Networks using Orthogonal Gradientscited by 36
  26. 2020Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Controlcited by 27
  27. 2018Decoupling Dynamics and Reward for Transfer Learningcited by 23
  28. 2019Action Semantics Network: Considering the Effects of Actions in Multiagent Systemscited by 19
  29. 2022Transfer RL across Observation Feature Spaces via Model-Based Regularizationcited by 18
  30. 2019Meta Reinforcement Learning with Task Embedding and Shared Policycited by 15
  31. 2020REPAINT: Knowledge Transfer in Deep Reinforcement Learningcited by 14
  32. 2021Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignmentcited by 12
  33. 2020An Efficient Transfer Learning Framework for Multiagent Reinforcement Learningcited by 11
  34. 2019Policy Distillation and Value Matching in Multiagent Reinforcement Learningcited by 10
  35. 2019Disentangled Skill Embeddings for Reinforcement Learningcited by 9
  36. 2021LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammatescited by 9
  37. 2018VPE: Variational Policy Embedding for Transfer Reinforcement Learningcited by 8
  38. 2021Learning Meta Representations for Agents in Multi-Agent Reinforcement Learningcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 7; 2025: 4.

14 papers.

Sort:

By area:

  1. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 10
  2. 2024Multi-Sender Persuasion: A Computational Perspectivecited by 9
  3. 2022Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Frameworkcited by 6
  4. 2024Roadmap on Incentive Compatibility for AI Alignment and Governance in Sociotechnical Systemscited by 6
  5. 2024Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaborationcited by 6
  6. 2024The Bandit Whisperer: Communication Learning for Restless Banditscited by 5
  7. 2024Hybrid Training for Enhanced Multi-task Generalization in Multi-agent Reinforcement Learningcited by 3
  8. 2025HiVA: Self-organized Hierarchical Variable Agent via Goal-driven Semantic-Topological Evolutioncited by 3
  9. 2025Equilibrium Policy Generalization: A Reinforcement Learning Framework for Cross-Graph Zero-Shot Generalization in Pursuit-Evasion Gamescited by 3
  10. 2023Learning Embeddings for Sequential Tasks Using Population of Agentscited by 0
  11. 2024CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Communitycited by 0
  12. 2024FlickerFusion: Intra-trajectory Domain Generalizing Multi-Agent RLcited by 0
2 more
  1. 2025Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Reviewcited by 0
  2. 2025Clutter Detection and Removal by Multi-Objective Analysis for Photographic Guidancecited by 0