# Video Processing from Electro-optical Sensors for Object Detection and Tracking in Maritime Environment: A Survey

arXiv: https://arxiv.org/abs/1611.05842 · HTML: https://mlsanity.com/paper/1611.05842/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2016.

## Built on by (most cited first, 30 of 47)

- [RPCANet: Deep Unfolding RPCA Based Infrared Small Target Detection](https://mlsanity.com/paper/2311.00917/) (2023)
- [Agents' Last Exam](https://mlsanity.com/paper/2606.05405/) (2026)
- [MODS -- A USV-oriented object detection and obstacle segmentation benchmark](https://mlsanity.com/paper/2105.02359/) (2021)
- [LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmark](https://mlsanity.com/paper/2308.09618/) (2023)
- [A Guide to Image and Video based Small Object Detection using Deep Learning : Case Study of Maritime Surveillance](https://mlsanity.com/paper/2207.12926/) (2022)
- [Survey on video anomaly detection in dynamic scenes with moving cameras](https://mlsanity.com/paper/2308.07050/) (2023)
- [An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needs](https://mlsanity.com/paper/2105.01342/) (2021)
- [Learning Maritime Obstacle Detection from Weak Annotations by Scaffolding](https://mlsanity.com/paper/2108.00564/) (2021)
- [MassMIND: Massachusetts Maritime INfrared Dataset](https://mlsanity.com/paper/2209.04097/) (2022)
- [M-SpecGene: Generalized Foundation Model for RGBT Multispectral Vision](https://mlsanity.com/paper/2507.16318/) (2025)
- [AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond](https://mlsanity.com/paper/2509.26636/) (2025)
- [A Survey on Infrared Image and Video Sets](https://mlsanity.com/paper/2203.08581/) (2022)
- [DeepSeaNet: Improving Underwater Object Detection using EfficientDet](https://mlsanity.com/paper/2306.06075/) (2023)
- [Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection](https://mlsanity.com/paper/2509.20745/) (2025)
- [eWaSR -- an embedded-compute-ready maritime obstacle detection network](https://mlsanity.com/paper/2304.11249/) (2023)
- [Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions](https://mlsanity.com/paper/2409.01500/) (2024)
- [SimuShips -- A High Resolution Simulation Dataset for Ship Detection with Precise Annotations](https://mlsanity.com/paper/2211.05237/) (2022)
- [AoSRNet: All-in-One Scene Recovery Networks via Multi-knowledge Integration](https://mlsanity.com/paper/2402.03738/) (2024)
- [Leveraging Graph and Deep Learning Uncertainties to Detect Anomalous Trajectories](https://mlsanity.com/paper/2107.01557/) (2021)
- [SeaDSC: A video-based unsupervised method for dynamic scene change detection in unmanned surface vehicles](https://mlsanity.com/paper/2311.11580/) (2023)
- [SafeSea: Synthetic Data Generation for Adverse & Low Probability Maritime Conditions](https://mlsanity.com/paper/2311.14764/) (2023)
- [MvKSR: Multi-view Knowledge-guided Scene Recovery for Hazy and Rainy Degradation](https://mlsanity.com/paper/2401.03800/) (2024)
- [Introducing VaDA: Novel Image Segmentation Model for Maritime Object Segmentation Using New Dataset](https://mlsanity.com/paper/2407.09005/) (2024)
- [Inland Waterway Object Detection in Multi-environment: Dataset and Approach](https://mlsanity.com/paper/2504.04835/) (2025)
- [Enhancing Maritime Object Detection in Real-Time with RT-DETR and Data Augmentation](https://mlsanity.com/paper/2510.07346/) (2025)
- [A Fast Horizon Detector and a New Annotated Dataset for Maritime Video Processing](https://mlsanity.com/paper/2110.13694/) (2021)
- [KOLOMVERSE: Korea open large-scale image dataset for object detection in the maritime universe](https://mlsanity.com/paper/2206.09885/) (2022)
- [A Transfer Learning-Based Approach to Marine Vessel Re-Identification](https://mlsanity.com/paper/2207.14500/) (2022)
- [Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons](https://mlsanity.com/paper/2311.07955/) (2023)
- [MVTD: A Benchmark Dataset for Maritime Visual Object Tracking](https://mlsanity.com/paper/2506.02866/) (2025)

## Abstract (arXiv)

We present a survey on maritime object detection and tracking approaches, which are essential for the development of a navigational system for autonomous ships. The electro-optical (EO) sensor considered here is a video camera that operates in the visible or the infrared spectra, which conventionally complement radar and sonar and have demonstrated effectiveness for situational awareness at sea has demonstrated its effectiveness over the last few years. This paper provides a comprehensive overview of various approaches of video processing for object detection and tracking in the maritime environment. We follow an approach-based taxonomy wherein the advantages and limitations of each approach are compared. The object detection system consists of the following modules: horizon detection, static background subtraction and foreground segmentation. Each of these has been studied extensively in maritime situations and has been shown to be challenging due to the presence of background motion especially due to waves and wakes. The main processes involved in object tracking include video frame registration, dynamic background subtraction, and the object tracking algorithm itself. The challenges for robust tracking arise due to camera motion, dynamic background and low contrast of tracked object, possibly due to environmental degradation. The survey also discusses multisensor approaches and commercial maritime systems that use EO sensors. The survey also highlights methods from computer vision research which hold promise to perform well in maritime EO data processing. Performance of several maritime and computer vision techniques is evaluated on newly proposed Singapore Maritime Dataset.