Multi-Agent Generative Adversarial Interactive Self-Imitation Learning for AUV Formation Control and Obstacle Avoidance
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
Multiple autonomous underwater vehicles (multi-AUV) can cooperatively accomplish tasks that a single AUV cannot complete. Recently, multi-agent reinforcement learning has been introduced to control of multi-AUV. However, designing efficient reward functions for various tasks of multi-AUV control is difficult or even impractical. Multi-agent generative adversarial imitation learning (MAGAIL) allows multi-AUV to learn from expert demonstration instead of pre-defined reward functions, but suffers from the deficiency of requiring optimal demonstrations and not surpassing provided expert demonstrations. This paper builds upon the MAGAIL algorithm by proposing multi-agent generative adversarial interactive self-imitation learning (MAGAISIL), which can facilitate AUVs to learn policies by gradually replacing the provided sub-optimal demonstrations with self-generated good trajectories selected by a human trainer. Our experimental results in a multi-AUV formation control and obstacle avoidance task on the Gazebo platform with AUV simulator of our lab show that AUVs trained via MAGAISIL can surpass the provided sub-optimal expert demonstrations and reach a performance close to or even better than MAGAIL with optimal demonstrations. Further results indicate that AUVs' policies trained via MAGAISIL can adapt to complex and different tasks as well as MAGAIL learning from optimal demonstrations.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 38 references that are in the map. First posted on arXiv 21 Jan 2024.
Builds on
10 of 38 references.
Closest ancestors · 2
Same direction, newest first
- 2018Generative Adversarial Self-Imitation Learning3 Dec
- 2018Multi-Agent Generative Adversarial Imitation Learning26 Jul
Other references · 8
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2016Generative Adversarial Imitation Learningcited by 1,597
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
- 2017A Brief Survey of Deep Reinforcement Learningcited by 483
- 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
- 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
- 2021Deep Reinforcement Learning for Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Medium Transitioncited by 9
- 2020Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehiclecited by 4
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Is FISHER All You Need in The Multi-AUV Underwater Target Tracking Task?
- 2019Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems
- 2025Ocean Diviner: A Diffusion-Augmented Reinforcement Learning Framework for AUV Robust Control in Underwater Tasks
- 2020ADAIL: Adaptive Adversarial Imitation Learning
- 2025SPLASH! Sample-efficient Preference-based inverse reinforcement learning for Long-horizon Adversarial tasks from Suboptimal Hierarchical demonstrations
- 2018OIL: Observational Imitation Learning
- 2025Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles
- 2025Long-Term Mapping of the Douro River Plume with Multi-Agent Reinforcement Learning