# Boosting Sample Efficiency and Generalization in Multi-agent Reinforcement Learning via Equivariance

arXiv: https://arxiv.org/abs/2410.02581 · HTML: https://mlsanity.com/paper/2410.02581/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Geometric and Physical Quantities Improve E(3) Equivariant Message Passing](https://mlsanity.com/paper/2110.02905/) (2021)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning](https://mlsanity.com/paper/2006.16908/) (2020)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Learning Transferable Cooperative Behavior in Multi-Agent Teams](https://mlsanity.com/paper/1906.01202/) (2019)

## Built on by (most cited first, 10 of 10)

- [Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning](https://mlsanity.com/paper/2510.03578/) (2025)
- [Robust Noise Attenuation via Adaptive Pooling of Transformer Outputs](https://mlsanity.com/paper/2506.09215/) (2025)
- [Closed-Loop Vision-Language Planning for Multi-Agent Coordination](https://mlsanity.com/paper/2502.10148/) (2025)
- [SPECTra: Scalable Multi-Agent Reinforcement Learning with Permutation-Free Networks](https://mlsanity.com/paper/2503.11726/) (2025)
- [PEnGUiN: Partially Equivariant Graph NeUral Networks for Sample Efficient MARL](https://mlsanity.com/paper/2503.15615/) (2025)
- [cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending](https://mlsanity.com/paper/2508.20818/) (2025)
- [Symmetries-enhanced Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.01136/) (2025)
- [Local-Canonicalization Equivariant Graph Neural Networks for Sample-Efficient and Generalizable Swarm Robot Control](https://mlsanity.com/paper/2509.14431/) (2025)
- [Reinforcement Learning Using known Invariances](https://mlsanity.com/paper/2511.03473/) (2025)
- [ARAC: Adaptive Regularized Multi-Agent Soft Actor-Critic in Graph-Structured Adversarial Games](https://mlsanity.com/paper/2511.08412/) (2025)

## Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) struggles with sample inefficiency and poor generalization [1]. These challenges are partially due to a lack of structure or inductive bias in the neural networks typically used in learning the policy. One such form of structure that is commonly observed in multi-agent scenarios is symmetry. The field of Geometric Deep Learning has developed Equivariant Graph Neural Networks (EGNN) that are equivariant (or symmetric) to rotations, translations, and reflections of nodes. Incorporating equivariance has been shown to improve learning efficiency and decrease error [ 2 ]. In this paper, we demonstrate that EGNNs improve the sample efficiency and generalization in MARL. However, we also show that a naive application of EGNNs to MARL results in poor early exploration due to a bias in the EGNN structure. To mitigate this bias, we present Exploration-enhanced Equivariant Graph Neural Networks or E2GN2. We compare E2GN2 to other common function approximators using common MARL benchmarks MPE and SMACv2. E2GN2 demonstrates a significant improvement in sample efficiency, greater final reward convergence, and a 2x-5x gain in over standard GNNs in our generalization tests. These results pave the way for more reliable and effective solutions in complex multi-agent systems.