# A Diver Attention Estimation Framework for Effective Underwater Human-Robot Interaction

arXiv: https://arxiv.org/abs/2209.14447 · HTML: https://mlsanity.com/paper/2209.14447/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Fine-Grained Head Pose Estimation Without Keypoints](https://mlsanity.com/paper/1710.00925/) (2017)
- [S3FD: Single Shot Scale-invariant Face Detector](https://mlsanity.com/paper/1708.05237/) (2017)
- [DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation](https://mlsanity.com/paper/1511.06645/) (2015)
- [DSFD: Dual Shot Face Detector](https://mlsanity.com/paper/1810.10220/) (2018)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs](https://mlsanity.com/paper/1907.05047/) (2019)
- [PyramidBox: A Context-assisted Single Shot Face Detector](https://mlsanity.com/paper/1803.07737/) (2018)
- [Simultaneous Enhancement and Super-Resolution of Underwater Imagery for Improved Visual Perception](https://mlsanity.com/paper/2002.01155/) (2020)
- [SSH: Single Stage Headless Face Detector](https://mlsanity.com/paper/1708.03979/) (2017)
- [Some Improvements on Deep Convolutional Neural Network Based Image Classification](https://mlsanity.com/paper/1312.5402/) (2013)
- [LFFD: A Light and Fast Face Detector for Edge Devices](https://mlsanity.com/paper/1904.10633/) (2019)
- [SVAM: Saliency-guided Visual Attention Modeling by Autonomous Underwater Robots](https://mlsanity.com/paper/2011.06252/) (2020)
- [Face Attention Network: An Effective Face Detector for the Occluded Faces](https://mlsanity.com/paper/1711.07246/) (2017)
- [Robust Gesture-Based Communication for Underwater Human-Robot Interaction in the context of Search and Rescue Diver Missions](https://mlsanity.com/paper/1810.07122/) (2018)
- [Underwater Image Super-Resolution using Deep Residual Multipliers](https://mlsanity.com/paper/1909.09437/) (2019)
- [Understanding Human Motion and Gestures for Underwater Human-Robot Collaboration](https://mlsanity.com/paper/1804.02479/) (2018)
- [Dynamic Reconfiguration of Mission Parameters in Underwater Human-Robot Collaboration](https://mlsanity.com/paper/1709.08772/) (2017)
- [Visual Diver Recognition for Underwater Human-Robot Collaboration](https://mlsanity.com/paper/1809.10201/) (2018)
- [Using Monocular Vision and Human Body Priors for AUVs to Autonomously Approach Divers](https://mlsanity.com/paper/2111.03712/) (2021)

## Abstract (arXiv)

Many underwater tasks, such as cable-and-wreckage inspection and search-and-rescue, can benefit from robust Human-Robot Interaction (HRI) capabilities. With the recent advancements in vision-based underwater HRI methods, Autonomous Underwater Vehicles (AUVs) have the capability to interact with their human partners without requiring assistance from a topside operator. However, in these methods, the AUV assumes that the diver is ready for interaction, while in reality, the diver may be distracted. In this paper, we attempt to address this problem by presenting a diver attention estimation framework for AUVs to autonomously determine the attentiveness of a diver, and developing a robot controller to allow the AUV to navigate and reorient itself with respect to the diver before initiating interaction. The core element of the framework is a deep convolutional neural network called DATT-Net. It is based on a pyramid structure that can exploit the geometric relations among 10 facial keypoints of a diver to estimate their head orientation, which we use as an indicator of attentiveness. Our on-the-bench experimental evaluations and real-world experiments during both closed- and open-water robot trials confirm the efficacy of the proposed framework.