# Decentralized Circle Formation Control for Fish-like Robots in the Real-world via Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.05293 · HTML: https://mlsanity.com/paper/2103.05293/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 9 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1905.05408/) (2019)
- [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://mlsanity.com/paper/1810.05587/) (2018)
- [Neighborhood Cognition Consistent Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1912.01160/) (2019)
- [Depth Control of Model-Free AUVs via Reinforcement Learning](https://mlsanity.com/paper/1711.08224/) (2017)

## Built on by (most cited first, 1 of 1)

- [Multi-Agent Guided Policy Optimization](https://mlsanity.com/paper/2507.18059/) (2025)

## Abstract (arXiv)

In this paper, the circle formation control problem is addressed for a group of cooperative underactuated fish-like robots involving unknown nonlinear dynamics and disturbances. Based on the reinforcement learning and cognitive consistency theory, we propose a decentralized controller without the knowledge of the dynamics of the fish-like robots. The proposed controller can be transferred from simulation to reality. It is only trained in our established simulation environment, and the trained controller can be deployed to real robots without any manual tuning. Simulation results confirm that the proposed model-free robust formation control method is scalable with respect to the group size of the robots and outperforms other representative RL algorithms. Several experiments in the real world verify the effectiveness of our RL-based approach for circle formation control.