mlsanityData to 14 Oct 2026

PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learning

arXiv 1911.00025 ↗First posted 31 Oct 2019cs.LG cs.CV stat.MLcited by 47

Not on the map. Why?

Abstract (arXiv)

Sample efficiency and scalability to a large number of agents are two important goals for multi-agent reinforcement learning systems. Recent works got us closer to those goals, addressing non-stationarity of the environment from a single agent's perspective by utilizing a deep net critic which depends on all observations and actions. The critic input concatenates agent observations and actions in a user-specified order. However, since deep nets aren't permutation invariant, a permuted input changes the critic output despite the environment remaining identical. To avoid this inefficiency, we propose a 'permutation invariant critic' (PIC), which yields identical output irrespective of the agent permutation. This consistent representation enables our model to scale to 30 times more agents and to achieve improvements of test episode reward between 15% to 50% on the challenging multi-agent particle environment (MPE).

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 13; 2023: 6; 2024: 3; 2025: 9; 2026: 4.

47 papers.

Sort:

By area:

  1. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 75
  2. 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 74
  3. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  4. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  5. 2021The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigationcited by 27
  6. 2021Multi-Agent MDP Homomorphic Networkscited by 25
  7. 2021GridToPix: Training Embodied Agents with Minimal Supervisioncited by 21
  8. 2022Maximum Class Separation as Inductive Bias in One Matrixcited by 21
  9. 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agentscited by 19
  10. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18
  11. 2022Equivariance Discovery by Learned Parameter-Sharingcited by 18
  12. 2022Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networkscited by 18
35 more
  1. 2024Group-Aware Coordination Graph for Multi-Agent Reinforcement Learningcited by 18
  2. 2022Equivariant Networks for Zero-Shot Coordinationcited by 16
  3. 2023Making Vision Transformers Truly Shift-Equivariantcited by 16
  4. 2023Truly Scale-Equivariant Deep Nets with Fourier Layerscited by 16
  5. 2021Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approachcited by 14
  6. 2022Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARLcited by 12
  7. 2022ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Rewardcited by 11
  8. 2022Agent-Temporal Attention for Reward Redistribution in Episodic Multi-Agent Reinforcement Learningcited by 9
  9. 2022Learning Control Admissibility Models with Graph Neural Networks for Multi-Agent Navigationcited by 8
  10. 2024Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learningcited by 7
  11. 2021Agent-Centric Representations for Multi-Agent Reinforcement Learningcited by 6
  12. 2022Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Frameworkcited by 6
  13. 2023Multi-Flow Transmission in Wireless Interference Networks: A Convergent Graph Learning Approachcited by 6
  14. 2023rm E(3)-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learningcited by 6
  15. 2022Communication-Efficient Actor-Critic Methods for Homogeneous Markov Gamescited by 5
  16. 2022Domain Adversarial Graph Convolutional Network Based on RSSI and Crowdsensing for Indoor Localizationcited by 5
  17. 2021Efficient Reinforcement Learning in Resource Allocation Problems Through Permutation Invariant Multi-task Learningcited by 3
  18. 2025Group Downsampling with Equivariant Anti-aliasingcited by 3
  19. 2025SPECTra: Scalable Multi-Agent Reinforcement Learning with Permutation-Free Networkscited by 2
  20. 2025Local Scale Equivariance with Latent Deep Equilibrium Canonicalizercited by 2
  21. 2025CLIPSym: Delving into Symmetry Detection with CLIPcited by 2
  22. 2024HARP: Human-Assisted Regrouping with Permutation Invariant Critic for Multi-Agent Reinforcement Learningcited by 1
  23. 2025Symmetries-enhanced Multi-Agent Reinforcement Learningcited by 1
  24. 2025Beyond Shallow Behavior: Task-Efficient Value-Based Multi-Task Offline MARL via Skill Discoverycited by 1
  25. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demoscited by 1
  26. 2021Contrastive Video-Language Segmentationcited by 0
  27. 2022RACA: Relation-Aware Credit Assignment for Ad-Hoc Cooperation in Multi-Agent Deep Reinforcement Learningcited by 0
  28. 2023Knowledge-Driven Multi-Agent Reinforcement Learning for Computation Offloading in Cybertwin-Enabled Internet of Vehiclescited by 0
  29. 2023Amortized Network Intervention to Steer the Excitatory Point Processescited by 0
  30. 2025Local-Canonicalization Equivariant Graph Neural Networks for Sample-Efficient and Generalizable Swarm Robot Controlcited by 0
  31. 2025Reinforcement Learning Using known Invariancescited by 0
  32. 2026Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networkscited by 0
  33. 2026Tunable Soft Equivariance with Guaranteescited by 0
  34. 2026PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillationcited by 0
  35. 2026CCKS: Consensus-based Communication and Knowledge Sharingcited by 0