mlsanityData to 7 Oct 2026

Multi-Agent MDP Homomorphic Networks

arXiv 2110.04495 ↗First posted 9 Oct 2021cs.LG cs.MAcited by 24

Not on the map. Why?

Abstract (arXiv)

This paper introduces Multi-Agent MDP Homomorphic Networks, a class of networks that allows distributed execution using only local information, yet is able to share experience between global symmetries in the joint state-action space of cooperative multi-agent systems. In cooperative multi-agent systems, complex symmetries arise between different configurations of the agents and their local observations. For example, consider a group of agents navigating: rotating the state globally results in a permutation of the optimal joint policy. Existing work on symmetries in single agent reinforcement learning can only be generalized to the fully centralized setting, because such approaches rely on the global symmetry in the full state-action spaces, and these can result in correspondences across agents. To encode such symmetries while still allowing distributed execution we propose a factorization that decomposes global symmetries into local transformations. Our proposed factorization allows for distributing the computation that enforces global symmetries over local agents and local interactions. We introduce a multi-agent equivariant policy network based on this factorization. We show empirically on symmetric multi-agent problems that globally symmetric distributable policies improve data efficiency compared to non-equivariant baselines.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 9 Oct 2021.

Builds on

28 of 43 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Group Equivariant Convolutional Networkscited by 1,154
  3. 2018Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point cloudscited by 735
  4. 2020SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networkscited by 602
  5. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  6. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  7. 2020Reinforcement Learning with Augmented Datacited by 452
  8. 2019General E(2)-Equivariant Steerable CNNscited by 441
  9. 2016Learning Multiagent Communication with Backpropagationcited by 355
  10. 20183D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Datacited by 348
  11. 2016Steerable CNNscited by 298
  12. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
16 more
  1. 2020"Other-Play" for Zero-Shot Coordinationcited by 194
  2. 2018Graph Convolutional Reinforcement Learningcited by 148
  3. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
  4. 2019Deep Coordination Graphscited by 96
  5. 2019A Survey on Traffic Signal Control Methodscited by 78
  6. 2020Neural Enhanced Belief Propagation on Factor Graphscited by 55
  7. 2020Symmetry-Aware Actor-Critic for 3D Molecular Designcited by 55
  8. 2019PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learningcited by 46
  9. 2018Inference in Probabilistic Graphical Models by Graph Neural Networkscited by 38
  10. 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
  11. 20183D G-CNNs for Pulmonary Nodule Detectioncited by 36
  12. 2020Group Equivariant Generative Adversarial Networkscited by 19
  13. 2019Augmenting learning using symmetry in a biologically-inspired domaincited by 4
  14. 2020Goal-conditioned Batch Reinforcement Learning for Rotation Invariant Locomotioncited by 4
  15. 2014Teaching Deep Convolutional Neural Networks to Play Gocited by 2
  16. 2015Exploiting Anonymity in Approximate Linear Programming: Scaling to Large Multiagent MDPs (Extended Version)cited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 7; 2024: 2; 2025: 7; 2026: 2.

24 papers.

Sort:

By area:

  1. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  2. 2022Maximum Class Separation as Inductive Bias in One Matrixcited by 21
  3. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16
  4. 2022Continual Vision-based Reinforcement Learning with Group Symmetriescited by 10
  5. 2025Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objectscited by 10
  6. 2023ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learningcited by 9
  7. 2023Policy Gradient Methods in the Presence of Symmetries and State Abstractionscited by 8
  8. 2022Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Frameworkcited by 6
  9. 2023Understanding when Dynamics-Invariant Data Augmentations Benefit Model-Free Reinforcement Learning Updatescited by 6
  10. 2024Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstractioncited by 6
  11. 2022Equivariant Reinforcement Learning for Quadrotor UAVcited by 4
  12. 2023rm E(3)-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learningcited by 4
12 more
  1. 2023Equivariant MuZerocited by 3
  2. 2023Enriching Disentanglement: From Logical Definitions to Quantitative Metricscited by 2
  3. 2024Reinforcement Learning with Euclidean Data Augmentation for State-Based Continuous Controlcited by 2
  4. 2025Improving Controller Generalization with Dimensionless Markov Decision Processescited by 2
  5. 2025PEnGUiN: Partially Equivariant Graph NeUral Networks for Sample Efficient MARLcited by 1
  6. 2025Learning Abstract World Models with a Group-Structured Latent Spacecited by 1
  7. 2026Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systemscited by 1
  8. 2023Centralized control for multi-agent RL in a complex Real-Time-Strategy gamecited by 0
  9. 2025Enhancing Traffic Signal Control through Model-based Reinforcement Learning and Policy Reusecited by 0
  10. 2025Symmetry-Guided Multi-Agent Inverse Reinforcement Learningcited by 0
  11. 2025Reinforcement Learning Using known Invariancescited by 0
  12. 2026ACPO: Agent-Chained Policy Optimization for Multi-Agent Reinforcement Learningcited by 0