mlsanityData to 14 Oct 2026

SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning

arXiv 2105.06228 ↗First posted 13 May 2021cs.MA cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

As one of the solutions to the decentralized partially observable Markov decision process (Dec-POMDP) problems, the value decomposition method has achieved significant results recently. However, most value decomposition methods require the fully observable state of the environment during training, but this is not feasible in some scenarios where only incomplete and noisy observations can be obtained. Therefore, we propose a novel value decomposition framework, named State Inference for value DEcomposition (SIDE), which eliminates the need to know the global state by simultaneously seeking solutions to the two problems of optimal control and state inference. SIDE can be extended to any value decomposition method to tackle partially observable problems. By comparing with the performance of different algorithms in StarCraft II micromanagement tasks, we verified that though without accessible states, SIDE can infer the current state that contributes to the reinforcement learning process based on past local observations and even achieve superior results to many baselines in some complex scenarios.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 13 May 2021.

Builds on

38 of 51 references.

References · 38

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2016Variational Graph Auto-Encoderscited by 1,382
  5. 2016HyperNetworkscited by 1,027
  6. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  7. 2019Solving Rubik's Cube with a Robot Handcited by 734
  8. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  9. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  10. 2019The StarCraft Multi-Agent Challengecited by 597
  11. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  12. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
26 more
  1. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  2. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  3. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  4. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  5. 2016Learning Multiagent Communication with Backpropagationcited by 366
  6. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  7. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  8. 2019MAVEN: Multi-Agent Variational Explorationcited by 185
  9. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 137
  10. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  11. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  12. 2016DESPOT: Online POMDP Planning with Regularizationcited by 106
  13. 2017Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Gamescited by 96
  14. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  15. 2019Symmetric Graph Convolutional Autoencoder for Unsupervised Graph Representation Learningcited by 94
  16. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 93
  17. 2017PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planningcited by 69
  18. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  19. 2018Particle Filter Networks with Application to Visual Localizationcited by 65
  20. 2018Differentiable Particle Filters: End-to-End Learning with Algorithmic Priorscited by 59
  21. 2018On Improving Deep Reinforcement Learning for POMDPscited by 50
  22. 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 47
  23. 2020R-MADDPG for Partially Observable Environments and Limited Communicationcited by 36
  24. 2020Discriminative Particle Filter Reinforcement Learning for Complex Partial Observationscited by 28
  25. 2021Neural Recursive Belief States in Multi-Agent Reinforcement Learningcited by 18
  26. 2019Keep It Simple: Graph Autoencoders Without Graph Convolutional Networkscited by 14

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 2; 2024: 2; 2025: 2.

7 papers.

  1. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  2. 2025QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?cited by 6
  3. 2024On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flowcited by 5
  4. 2025Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Explorationcited by 3
  5. 2023Centralized Cooperative Exploration Policy for Continuous Control Taskscited by 1
  6. 2023Latent Interactive A2C for Improved RL in Open Many-Agent Systemscited by 0
  7. 2024Multi-agent Off-policy Actor-Critic Reinforcement Learning for Partially Observable Environmentscited by 0