# Scalable Reinforcement Learning Policies for Multi-Agent Control

arXiv: https://arxiv.org/abs/2011.08055 · HTML: https://mlsanity.com/paper/2011.08055/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Exploration](https://mlsanity.com/paper/2301.03398/) (2023)
- [Learning Continuous Control Policies for Information-Theoretic Active Perception](https://mlsanity.com/paper/2209.12427/) (2022)
- [Policy Learning for Active Target Tracking over Continuous SE(3) Trajectories](https://mlsanity.com/paper/2212.01498/) (2022)
- [Robust Noise Attenuation via Adaptive Pooling of Transformer Outputs](https://mlsanity.com/paper/2506.09215/) (2025)
- [Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems](https://mlsanity.com/paper/2212.06357/) (2022)
- [Strategic Maneuver and Disruption with Reinforcement Learning Approaches for Multi-Agent Coordination](https://mlsanity.com/paper/2203.09565/) (2022)
- [Active Classification of Moving Targets with Learned Control Policies](https://mlsanity.com/paper/2212.03068/) (2022)
- [Task-priority Intermediated Hierarchical Distributed Policies: Reinforcement Learning of Adaptive Multi-robot Cooperative Transport](https://mlsanity.com/paper/2404.02362/) (2024)
- [Reinforcement Learning of Multi-robot Task Allocation for Multi-object Transportation with Infeasible Tasks](https://mlsanity.com/paper/2404.11817/) (2024)
- [Active Scout: Multi-Target Tracking Using Neural Radiance Fields in Dense Urban Environments](https://mlsanity.com/paper/2406.07431/) (2024)
- [State-of-the-art in Robot Learning for Multi-Robot Collaboration: A Comprehensive Survey](https://mlsanity.com/paper/2408.11822/) (2024)
- [Collaborative State Fusion in Partially Known Multi-agent Environments](https://mlsanity.com/paper/2410.15137/) (2024)

## Abstract (arXiv)

We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observations (range and bearing) about targets which move using fixed, unknown policies. An attention mechanism is used to parameterize the value function of the agents; this mechanism allows us to handle an arbitrary number of targets. Entropy-regularized off-policy RL methods are used to train a stochastic policy, and we discuss how it enables a hedging behavior between pursuers that leads to a weak form of cooperation in spite of completely decentralized control execution. We further develop a masking heuristic that allows training on smaller problems with few pursuers-targets and execution on much larger problems. Thorough simulation experiments, ablation studies, and comparisons to state of the art algorithms are performed to study the scalability of the approach and robustness of performance to varying numbers of agents and targets.