# Is FISHER All You Need in The Multi-AUV Underwater Target Tracking Task?

arXiv: https://arxiv.org/abs/2412.03959 · HTML: https://mlsanity.com/paper/2412.03959/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models](https://mlsanity.com/paper/2201.03544/) (2022)
- [Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism](https://mlsanity.com/paper/2103.12021/) (2021)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [Efficient Exploration via State Marginal Matching](https://mlsanity.com/paper/1906.05274/) (2019)
- [Generalized Decision Transformer for Offline Hindsight Information Matching](https://mlsanity.com/paper/2111.10364/) (2021)
- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [RORL: Robust Offline Reinforcement Learning via Conservative Smoothing](https://mlsanity.com/paper/2206.02829/) (2022)
- [Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning](https://mlsanity.com/paper/1812.05027/) (2018)
- [A Unified Framework for Guiding Generative AI with Wireless Perception in Resource Constrained Mobile Edge Networks](https://mlsanity.com/paper/2309.01426/) (2023)
- [Generative Artificial Intelligence Assisted Wireless Sensing: Human Flow Detection in Practical Communication Environments](https://mlsanity.com/paper/2404.14140/) (2024)
- [Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation](https://mlsanity.com/paper/2312.09844/) (2023)
- [Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement](https://mlsanity.com/paper/2409.02428/) (2024)
- [A Study of Gradient Descent Schemes for General-Sum Stochastic Games](https://mlsanity.com/paper/1507.00093/) (2015)
- [Multi-AUV Cooperative Underwater Multi-Target Tracking Based on Dynamic-Switching-enabled Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2404.13654/) (2024)
- [Sample Efficient Learning of Path Following and Obstacle Avoidance Behavior for Quadrotors](https://mlsanity.com/paper/1906.12082/) (2019)

## Built on by (most cited first, 3 of 3)

- [OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval](https://mlsanity.com/paper/2507.05631/) (2025)
- [Never too Prim to Swim: An LLM-Enhanced RL-based Adaptive S-Surface Controller for AUVs under Extreme Sea Conditions](https://mlsanity.com/paper/2503.00527/) (2025)
- [Task-Semantic Graph-Driven Distributed Agent Networking for Underwater Target Tracking](https://mlsanity.com/paper/2605.15528/) (2026)

## Abstract (arXiv)

It is significant to employ multiple autonomous underwater vehicles (AUVs) to execute the underwater target tracking task collaboratively. However, it's pretty challenging to meet various prerequisites utilizing traditional control methods. Therefore, we propose an effective two-stage learning from demonstrations training framework, FISHER, to highlight the adaptability of reinforcement learning (RL) methods in the multi-AUV underwater target tracking task, while addressing its limitations such as extensive requirements for environmental interactions and the challenges in designing reward functions. The first stage utilizes imitation learning (IL) to realize policy improvement and generate offline datasets. To be specific, we introduce multi-agent discriminator-actor-critic based on improvements of the generative adversarial IL algorithm and multi-agent IL optimization objective derived from the Nash equilibrium condition. Then in the second stage, we develop multi-agent independent generalized decision transformer, which analyzes the latent representation to match the future states of high-quality samples rather than reward function, attaining further enhanced policies capable of handling various scenarios. Besides, we propose a simulation to simulation demonstration generation procedure to facilitate the generation of expert demonstrations in underwater environments, which capitalizes on traditional control methods and can easily accomplish the domain transfer to obtain demonstrations. Extensive simulation experiments from multiple scenarios showcase that FISHER possesses strong stability, multi-task performance and capability of generalization.