# Robust Gesture-Based Communication for Underwater Human-Robot Interaction in the context of Search and Rescue Diver Missions

arXiv: https://arxiv.org/abs/1810.07122 · HTML: https://mlsanity.com/paper/1810.07122/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2018.

## Built on by (most cited first, 14 of 14)

- [AQUA20: A Benchmark Dataset for Underwater Species Classification under Challenging Conditions](https://mlsanity.com/paper/2506.17455/) (2025)
- [DeepSeeColor: Realtime Adaptive Color Correction for Autonomous Underwater Vehicles via Deep Learning Methods](https://mlsanity.com/paper/2303.04025/) (2023)
- [Underwater Object Classification and Detection: first results and open challenges](https://mlsanity.com/paper/2201.00977/) (2022)
- [NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction](https://mlsanity.com/paper/2403.02274/) (2024)
- [The Common Objects Underwater (COU) Dataset for Robust Underwater Object Detection](https://mlsanity.com/paper/2502.20651/) (2025)
- [Underwater inspection and intervention dataset](https://mlsanity.com/paper/2107.13628/) (2021)
- [Pose Estimation from Camera Images for Underwater Inspection](https://mlsanity.com/paper/2407.16961/) (2024)
- [A Diver Attention Estimation Framework for Effective Underwater Human-Robot Interaction](https://mlsanity.com/paper/2209.14447/) (2022)
- [Interpretable Underwater Diver Gesture Recognition](https://mlsanity.com/paper/2312.04874/) (2023)
- [Zero-Shot Underwater Gesture Recognition](https://mlsanity.com/paper/2407.14103/) (2024)
- [Color Information-Based Automated Mask Generation for Detecting Underwater Atypical Glare Areas](https://mlsanity.com/paper/2502.16538/) (2025)
- [NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction](https://mlsanity.com/paper/2502.16718/) (2025)
- [ShellfishNet: A Domain-Specific Benchmark for Visual Recognition of Marine Molluscs](https://mlsanity.com/paper/2605.07338/) (2026)
- [Semantically-Aware Diver Activity Recognition Framework for Effective Underwater Multi-Human-Robot Collaboration](https://mlsanity.com/paper/2606.12374/) (2026)

## Abstract (arXiv)

We propose a robust gesture-based communication pipeline for divers to instruct an Autonomous Underwater Vehicle (AUV) to assist them in performing high-risk tasks and helping in case of emergency. A gesture communication language (CADDIAN) is developed, based on consolidated and standardized diver gestures, including an alphabet, syntax and semantics, ensuring a logical consistency. A hierarchical classification approach is introduced for hand gesture recognition based on stereo imagery and multi-descriptor aggregation to specifically cope with underwater image artifacts, e.g. light backscatter or color attenuation. Once the classification task is finished, a syntax check is performed to filter out invalid command sequences sent by the diver or generated by errors in the classifier. Throughout this process, the diver receives constant feedback from an underwater tablet to acknowledge or abort the mission at any time. The objective is to prevent the AUV from executing unnecessary, infeasible or potentially harmful motions. Experimental results under different environmental conditions in archaeological exploration and bridge inspection applications show that the system performs well in the field.