# Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation

arXiv: https://arxiv.org/abs/1802.07956 · HTML: https://mlsanity.com/paper/1802.07956/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2018.

## Built on by (most cited first, 21 of 21)

- [WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfaces](https://mlsanity.com/paper/2307.06505/) (2023)
- [MODS -- A USV-oriented object detection and obstacle segmentation benchmark](https://mlsanity.com/paper/2105.02359/) (2021)
- [LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmark](https://mlsanity.com/paper/2308.09618/) (2023)
- [1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results](https://mlsanity.com/paper/2211.13508/) (2022)
- [WaterVG: Waterway Visual Grounding based on Text-Guided Vision and mmWave Radar](https://mlsanity.com/paper/2403.12686/) (2024)
- [Survey on video anomaly detection in dynamic scenes with moving cameras](https://mlsanity.com/paper/2308.07050/) (2023)
- [From Screens to Scenes: A Survey of Embodied AI in Healthcare](https://mlsanity.com/paper/2501.07468/) (2025)
- [Tiny Obstacle Discovery by Occlusion-Aware Multilayer Regression](https://mlsanity.com/paper/2111.09204/) (2021)
- [Learning Maritime Obstacle Detection from Weak Annotations by Scaffolding](https://mlsanity.com/paper/2108.00564/) (2021)
- [NanoMVG: USV-Centric Low-Power Multi-Task Visual Grounding based on Prompt-Guided Camera and 4D mmWave Radar](https://mlsanity.com/paper/2408.17207/) (2024)
- [Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles](https://mlsanity.com/paper/2307.09027/) (2023)
- [SeaDSC: A video-based unsupervised method for dynamic scene change detection in unmanned surface vehicles](https://mlsanity.com/paper/2311.11580/) (2023)
- [SeePerSea: Multi-modal Perception Dataset of In-water Objects for Autonomous Surface Vehicles](https://mlsanity.com/paper/2404.18411/) (2024)
- [Introducing VaDA: Novel Image Segmentation Model for Maritime Object Segmentation Using New Dataset](https://mlsanity.com/paper/2407.09005/) (2024)
- [Real-Time Fusion of Visual and Chart Data for Enhanced Maritime Vision](https://mlsanity.com/paper/2507.13880/) (2025)
- [Efficient Multi-branch Segmentation Network for Situation Awareness in Autonomous Navigation](https://mlsanity.com/paper/2404.00366/) (2024)
- [MULTIAQUA: A multimodal maritime dataset and robust training strategies for multimodal semantic segmentation](https://mlsanity.com/paper/2512.17450/) (2025)
- [Quality-Driven and Diversity-Aware Sample Expansion for Robust Marine Obstacle Segmentation](https://mlsanity.com/paper/2512.13970/) (2025)
- [WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agents](https://mlsanity.com/paper/2602.22923/) (2026)
- [Unified Restoration-Perception Learning: Maritime Infrared-Visible Image Fusion and Segmentation](https://mlsanity.com/paper/2603.28414/) (2026)
- [IceHorizon: A Dataset for Horizon Detection in Ice-Covered Maritime Environments and Comparative Evaluation of Detection Methods](https://mlsanity.com/paper/2608.07018/) (2026)

## Abstract (arXiv)

A new obstacle detection algorithm for unmanned surface vehicles (USVs) is presented. A state-of-the-art graphical model for semantic segmentation is extended to incorporate boat pitch and roll measurements from the on-board inertial measurement unit (IMU), and a stereo verification algorithm that consolidates tentative detections obtained from the segmentation is proposed. The IMU readings are used to estimate the location of horizon line in the image, which automatically adjusts the priors in the probabilistic semantic segmentation model. We derive the equations for projecting the horizon into images, propose an efficient optimization algorithm for the extended graphical model, and offer a practical IMU-camera-USV calibration procedure. Using an USV equipped with multiple synchronized sensors, we captured a new challenging multi-modal dataset, and annotated its images with water edge and obstacles. Experimental results show that the proposed algorithm significantly outperforms the state of the art, with nearly 30% improvement in water-edge detection accuracy, an over 21% reduction of false positive rate, an almost 60% reduction of false negative rate, and an over 65% increase of true positive rate, while its Matlab implementation runs in real-time.