WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agents
Not on the map. Why?
Abstract (arXiv)
While autonomous navigation has achieved remarkable success in passive perception (e.g., object detection and segmentation), it remains fundamentally constrained by a void in knowledge-driven, interactive environmental cognition. In the high-stakes domain of maritime navigation, the ability to bridge the gap between raw visual perception and complex cognitive reasoning is not merely an enhancement but a critical prerequisite for Autonomous Surface Vessels to execute safe and precise maneuvers. To this end, we present WaterVideoQA, the first large-scale, comprehensive Video Question Answering benchmark specifically engineered for all-waterway environments. This benchmark encompasses 3,029 video clips across six distinct waterway categories, integrating multifaceted variables such as volatile lighting and dynamic weather to rigorously stress-test ASV capabilities across a five-tier hierarchical cognitive framework. Furthermore, we introduce NaviMind, a pioneering multi-agent neuro-symbolic system designed for open-ended maritime reasoning. By synergizing Adaptive Semantic Routing, Situation-Aware Hierarchical Reasoning, and Autonomous Self-Reflective Verification, NaviMind transitions ASVs from superficial pattern matching to regulation-compliant, interpretable decision-making. Experimental results demonstrate that our framework significantly transcends existing baselines, establishing a new paradigm for intelligent, trustworthy interaction in dynamic maritime environments.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 26 Feb 2026.
Builds on
22 of 39 references.
References · 22
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2023VideoChat: Chat-Centric Video Understandingcited by 851
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
- 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
- 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
- 2022Video Transformers: A Surveycited by 63
- 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
- 2015Fast image-based obstacle detection from unmanned surface vehiclescited by 23
- 2018Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentationcited by 21
- 2025Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advancescited by 21
- 2023LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmarkcited by 19
10 more
- 2023Achelous: A Fast Unified Water-surface Panoptic Perception Framework based on Fusion of Monocular Camera and 4D mmWave Radarcited by 16
- 2026AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wildcited by 15
- 2024WaterVG: Waterway Visual Grounding based on Text-Guided Vision and mmWave Radarcited by 13
- 2023ASY-VRNet: Waterway Panoptic Driving Perception Model based on Asymmetric Fair Fusion of Vision and 4D mmWave Radarcited by 11
- 2024NanoMVG: USV-Centric Low-Power Multi-Task Visual Grounding based on Prompt-Guided Camera and 4D mmWave Radarcited by 7
- 2025Cognitive Disentanglement for Referring Multi-Object Trackingcited by 6
- 2025USVTrack: USV-Based 4D Radar-Camera Tracking Dataset for Autonomous Driving in Inland Waterwayscited by 4
- 2024A comprehensive review of datasets and deep learning techniques for vision in Unmanned Surface Vehiclescited by 2
- 2025Inland Waterway Object Detection in Multi-environment: Dataset and Approachcited by 2
- 2025CaRaFFusion: Improving 2D Semantic Segmentation with Camera-Radar Point Cloud Fusion and Zero-Shot Image Inpaintingcited by 2
Built on by
Not cited yet.