# Asynchronous Trajectory Matching-Based Multimodal Maritime Data Fusion for Vessel Traffic Surveillance in Inland Waterways

arXiv: https://arxiv.org/abs/2302.11283 · HTML: https://mlsanity.com/paper/2302.11283/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 52 references that are in the map.
- First posted on arXiv 22 Feb 2023.

## Builds on (most cited first, 13 of 13)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [FSSD: Feature Fusion Single Shot Multibox Detector](https://mlsanity.com/paper/1712.00960/) (2017)
- [MODS -- A USV-oriented object detection and obstacle segmentation benchmark](https://mlsanity.com/paper/2105.02359/) (2021)
- [Target detection in synthetic aperture radar imagery: a state-of-the-art survey](https://mlsanity.com/paper/1804.04719/) (2018)

## Built on by (most cited first, 13 of 13)

- [WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfaces](https://mlsanity.com/paper/2307.06505/) (2023)
- [Agents' Last Exam](https://mlsanity.com/paper/2606.05405/) (2026)
- [WaterVG: Waterway Visual Grounding based on Text-Guided Vision and mmWave Radar](https://mlsanity.com/paper/2403.12686/) (2024)
- [AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond](https://mlsanity.com/paper/2509.26636/) (2025)
- [Neptune-X: Active X-to-Maritime Generation for Universal Maritime Object Detection](https://mlsanity.com/paper/2509.20745/) (2025)
- [A Survey of Distance-Based Vessel Trajectory Clustering: Data Pre-processing, Methodologies, Applications, and Experimental Evaluation](https://mlsanity.com/paper/2407.11084/) (2024)
- [Graph Learning-Driven Multi-Vessel Association: Fusing Multimodal Data for Maritime Intelligence](https://mlsanity.com/paper/2504.09197/) (2025)
- [Predicting Barge Presence and Quantity on Inland Waterways using Vessel Tracking Data: A Machine Learning Approach](https://mlsanity.com/paper/2501.00615/) (2024)
- [WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objects](https://mlsanity.com/paper/2604.07759/) (2026)
- [CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence](https://mlsanity.com/paper/2605.26524/) (2026)
- [NavEYE: Vision-Centered Multi-Sensor Fusion-Based Situational Awareness System for Intelligent Surface Vehicles](https://mlsanity.com/paper/2607.03915/) (2026)
- [Preventing Adversarial AI Attacks Against Autonomous Situational Awareness: A Maritime Case Study](https://mlsanity.com/paper/2505.21609/) (2025)
- [Multi-Modal Graph Neural Network-Inspired Vessel Identification and Tracking for UAV-Based Maritime Surveillance](https://mlsanity.com/paper/2610.13220/) (2026)

## Abstract (arXiv)

The automatic identification system (AIS) and video cameras have been widely exploited for vessel traffic surveillance in inland waterways. The AIS data could provide the vessel identity and dynamic information on vessel position and movements. In contrast, the video data could describe the visual appearances of moving vessels, but without knowing the information on identity, position and movements, etc. To further improve vessel traffic surveillance, it becomes necessary to fuse the AIS and video data to simultaneously capture the visual features, identity and dynamic information for the vessels of interest. However, traditional data fusion methods easily suffer from several potential limitations, e.g., asynchronous messages, missing data, random outliers, etc. In this work, we first extract the AIS- and video-based vessel trajectories, and then propose a deep learning-enabled asynchronous trajectory matching method (named DeepSORVF) to fuse the AIS-based vessel information with the corresponding visual targets. In addition, by combining the AIS- and video-based movement features, we also present a prior knowledge-driven anti-occlusion method to yield accurate and robust vessel tracking results under occlusion conditions. To validate the efficacy of our DeepSORVF, we have also constructed a new benchmark dataset (termed FVessel) for vessel detection, tracking, and data fusion. It consists of many videos and the corresponding AIS data collected in various weather conditions and locations. The experimental results have demonstrated that our method is capable of guaranteeing high-reliable data fusion and anti-occlusion vessel tracking.