mlsanityData to 7 Oct 2026

Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles

arXiv 2505.08222 ↗First posted 13 May 2025cs.RO cs.AI cs.DCcited by 0

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Autonomous vehicles (AVs) offer a cost-effective solution for scientific missions such as underwater tracking. Reinforcement learning (RL) has emerged as a powerful method for controlling AVs, but scaling to fleets (essential for multi-target tracking or rapidly moving targets) is challenging. Multi-Agent RL (MARL) is notoriously sample-inefficient, and while high-fidelity simulators like Gazebo's LRAUV provide up to 100x faster-than-real-time single-robot simulations, they offer little speedup in multi-vehicle scenarios, making MARL training impractical. Yet, high-fidelity simulation is crucial to test complex policies and close the sim-to-real gap. To address these limitations, we develop a GPU-accelerated environment that achieves up to 30,000x speedup over Gazebo while preserving its dynamics. This enables fast, end-to-end GPU training and seamless transfer to Gazebo for evaluation. We also introduce a Transformer-based architecture (TransfMAPPO) that learns policies invariant to fleet size and number of targets, enabling curriculum learning to train larger fleets on increasingly complex scenarios. After large-scale GPU training, we perform extensive evaluations in Gazebo, showing our method maintains tracking errors below 5m even with multiple fast-moving targets.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 17 of its 37 references that are in the map. First posted on arXiv 13 May 2025.

Builds on

17 of 37 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  3. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  4. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  6. 2019Model-Based Reinforcement Learning for Ataricited by 495
  7. 2022Transformers are Sample-Efficient World Modelscited by 317
  8. 2022Robot Operating System 2: Design, Architecture, and Uses In The Wildcited by 190
  9. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  10. 2024Simplifying Deep Temporal Difference Learningcited by 71
  11. 2018Learning and Querying Fast Generative Models for Reinforcement Learningcited by 48
  12. 2019From Few to More: Large-scale Dynamic Multiagent Curriculum Learningcited by 36
5 more
  1. 2022Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregationcited by 34
  2. 2023TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problemscited by 9
  3. 2024Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systemscited by 3
  4. 2024Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learningcited by 2
  5. 2023From Concept to Field Tests: Accelerated Development of Multi-AUV Missions Using a High-Fidelity Faster-than-Real-Time Simulatorcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion-Guided Cooperative Policy Learning for Target Tracking Based on Underwater Mobile Agent Networks
  2. 2026Multi-AUV Ad-hoc network-based Target Tracking: A Value Gradient Guidance Multi-Agent Diffusion Reinforcement Learning Approach
  3. 2024Is FISHER All You Need in The Multi-AUV Underwater Target Tracking Task?
  4. 2023A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles
  5. 2025Ocean Diviner: A Diffusion-Augmented Reinforcement Learning Framework for AUV Robust Control in Underwater Tasks ↗
  6. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
  7. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  8. 2026NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation