# R-MADDPG for Partially Observable Environments and Limited Communication

arXiv: https://arxiv.org/abs/2002.06684 · HTML: https://mlsanity.com/paper/2002.06684/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2020.

## Built on by (most cited first, 30 of 36)

- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Unbiased Asymmetric Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2105.11674/) (2021)
- [Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence](https://mlsanity.com/paper/2105.01798/) (2021)
- [Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real](https://mlsanity.com/paper/2203.11653/) (2022)
- [Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning](https://mlsanity.com/paper/2104.09936/) (2021)
- [From Explicit Communication to Tacit Cooperation:A Novel Paradigm for Cooperative MARL](https://mlsanity.com/paper/2304.14656/) (2023)
- [Informed POMDP: Leveraging Additional Information in Model-Based RL](https://mlsanity.com/paper/2306.11488/) (2023)
- [A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2201.01221/) (2022)
- [SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2105.06228/) (2021)
- [Deep reinforcement learning of event-triggered communication and control for multi-agent cooperative transport](https://mlsanity.com/paper/2103.15260/) (2021)
- [On Diffusion Models for Multi-Agent Partial Observability: Shared Attractors, Error Bounds, and Composite Flow](https://mlsanity.com/paper/2410.13953/) (2024)
- [Cochain Perspectives on Temporal-Difference Signals for Learning Beyond Markov Dynamics](https://mlsanity.com/paper/2602.06939/) (2026)
- [Beyond Local Views: Global State Inference with Diffusion Models for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2408.09501/) (2024)
- [Minimizing Communication while Maximizing Performance in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.08482/) (2021)
- [Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2108.04087/) (2021)
- [Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.06274/) (2022)
- [Non-Stationary Policy Learning for Multi-Timescale Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2307.08794/) (2023)
- [Hierarchical RNNs-Based Transformers MADDPG for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/2105.04888/) (2021)
- [MACRPO: Multi-Agent Cooperative Recurrent Policy Optimization](https://mlsanity.com/paper/2109.00882/) (2021)
- [Soft Hierarchical Graph Recurrent Networks for Many-Agent Partially Observable Environments](https://mlsanity.com/paper/2109.02032/) (2021)
- [A Policy Resonance Approach to Solve the Problem of Responsibility Diffusion in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2208.07753/) (2022)
- [Learning Practical Communication Strategies in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.01288/) (2022)
- [Towards Fault Tolerance in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2412.00534/) (2024)
- [Belief States for Cooperative Multi-Agent Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2504.08417/) (2025)
- [Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels](https://mlsanity.com/paper/2101.10369/) (2021)
- [Local Advantage Networks for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2112.12458/) (2021)
- [A New Approach to Training Multiple Cooperative Agents for Autonomous Driving](https://mlsanity.com/paper/2209.02157/) (2022)
- [Multi-robot Social-aware Cooperative Planning in Pedestrian Environments Using Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2211.15901/) (2022)

## Abstract (arXiv)

There are several real-world tasks that would benefit from applying multiagent reinforcement learning (MARL) algorithms, including the coordination among self-driving cars. The real world has challenging conditions for multiagent learning systems, such as its partial observable and nonstationary nature. Moreover, if agents must share a limited resource (e.g. network bandwidth) they must all learn how to coordinate resource use. This paper introduces a deep recurrent multiagent actor-critic framework (R-MADDPG) for handling multiagent coordination under partial observable set-tings and limited communication. We investigate recurrency effects on performance and communication use of a team of agents. We demonstrate that the resulting framework learns time dependencies for sharing missing observations, handling resource limitations, and developing different communication patterns among agents.