# Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2111.07334 · HTML: https://mlsanity.com/paper/2111.07334/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 14 Nov 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Multi-task Deep Reinforcement Learning with PopArt](https://mlsanity.com/paper/1809.04474/) (2018)
- [A New Framework for Multi-Agent Reinforcement Learning -- Centralized Training and Exploration with Decentralized Execution via Policy Distillation](https://mlsanity.com/paper/1910.09152/) (2019)
- [Distillation Strategies for Proximal Policy Optimization](https://mlsanity.com/paper/1901.08128/) (2019)
- [Adaptive Leader-Follower Formation Control and Obstacle Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1911.06882/) (2019)
- [Distributed stabilization control of rigid formations with prescribed orientation](https://mlsanity.com/paper/1606.03199/) (2016)
- [A drl based distributed formation control scheme with stream based collision avoidance](https://mlsanity.com/paper/2109.03037/) (2021)

## Built on by (most cited first, 7 of 7)

- [RALLY: Role-Adaptive LLM-Driven Yoked Navigation for Agentic UAV Swarms](https://mlsanity.com/paper/2507.01378/) (2025)
- [Decentralized Consensus Inference-based Hierarchical Reinforcement Learning for Multi-Constrained UAV Pursuit-Evasion Game](https://mlsanity.com/paper/2506.18126/) (2025)
- [Decentralized Adaptive Formation via Consensus-Oriented Multi-Agent Communication](https://mlsanity.com/paper/2307.12287/) (2023)
- [Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Range](https://mlsanity.com/paper/2403.03055/) (2024)
- [Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent System](https://mlsanity.com/paper/2501.13727/) (2025)
- [Multi-Agent Reinforcement Learning-Based UAV Pathfinding for Obstacle Avoidance in Stochastic Environment](https://mlsanity.com/paper/2310.16659/) (2023)
- [EFLUX: Elastic Multi-Robot Formation Navigation and Adaptation with Agentic LLMs](https://mlsanity.com/paper/2607.12050/) (2026)

## Abstract (arXiv)

Multi-agent formation as well as obstacle avoidance is one of the most actively studied topics in the field of multi-agent systems. Although some classic controllers like model predictive control (MPC) and fuzzy control achieve a certain measure of success, most of them require precise global information which is not accessible in harsh environments. On the other hand, some reinforcement learning (RL) based approaches adopt the leader-follower structure to organize different agents' behaviors, which sacrifices the collaboration between agents thus suffering from bottlenecks in maneuverability and robustness. In this paper, we propose a distributed formation and obstacle avoidance method based on multi-agent reinforcement learning (MARL). Agents in our system only utilize local and relative information to make decisions and control themselves distributively. Agent in the multi-agent system will reorganize themselves into a new topology quickly in case that any of them is disconnected. Our method achieves better performance regarding formation error, formation convergence rate and on-par success rate of obstacle avoidance compared with baselines (both classic control methods and another RL-based method). The feasibility of our method is verified by both simulation and hardware implementation with Ackermann-steering vehicles.