mlsanityData to 7 Oct 2026

WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agents

arXiv 2602.22923 ↗First posted 26 Feb 2026cs.CV cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

While autonomous navigation has achieved remarkable success in passive perception (e.g., object detection and segmentation), it remains fundamentally constrained by a void in knowledge-driven, interactive environmental cognition. In the high-stakes domain of maritime navigation, the ability to bridge the gap between raw visual perception and complex cognitive reasoning is not merely an enhancement but a critical prerequisite for Autonomous Surface Vessels to execute safe and precise maneuvers. To this end, we present WaterVideoQA, the first large-scale, comprehensive Video Question Answering benchmark specifically engineered for all-waterway environments. This benchmark encompasses 3,029 video clips across six distinct waterway categories, integrating multifaceted variables such as volatile lighting and dynamic weather to rigorously stress-test ASV capabilities across a five-tier hierarchical cognitive framework. Furthermore, we introduce NaviMind, a pioneering multi-agent neuro-symbolic system designed for open-ended maritime reasoning. By synergizing Adaptive Semantic Routing, Situation-Aware Hierarchical Reasoning, and Autonomous Self-Reflective Verification, NaviMind transitions ASVs from superficial pattern matching to regulation-compliant, interpretable decision-making. Experimental results demonstrate that our framework significantly transcends existing baselines, establishing a new paradigm for intelligent, trustworthy interaction in dynamic maritime environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 39 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

22 of 39 references.

References · 22

  1. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  2. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  3. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  4. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  5. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
  6. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
  7. 2022Video Transformers: A Surveycited by 63
  8. 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
  9. 2015Fast image-based obstacle detection from unmanned surface vehiclescited by 23
  10. 2018Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentationcited by 21
  11. 2025Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advancescited by 21
  12. 2023LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Benchmarkcited by 19
10 more
  1. 2023Achelous: A Fast Unified Water-surface Panoptic Perception Framework based on Fusion of Monocular Camera and 4D mmWave Radarcited by 16
  2. 2026AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wildcited by 15
  3. 2024WaterVG: Waterway Visual Grounding based on Text-Guided Vision and mmWave Radarcited by 13
  4. 2023ASY-VRNet: Waterway Panoptic Driving Perception Model based on Asymmetric Fair Fusion of Vision and 4D mmWave Radarcited by 11
  5. 2024NanoMVG: USV-Centric Low-Power Multi-Task Visual Grounding based on Prompt-Guided Camera and 4D mmWave Radarcited by 7
  6. 2025Cognitive Disentanglement for Referring Multi-Object Trackingcited by 6
  7. 2025USVTrack: USV-Based 4D Radar-Camera Tracking Dataset for Autonomous Driving in Inland Waterwayscited by 4
  8. 2024A comprehensive review of datasets and deep learning techniques for vision in Unmanned Surface Vehiclescited by 2
  9. 2025Inland Waterway Object Detection in Multi-environment: Dataset and Approachcited by 2
  10. 2025CaRaFFusion: Improving 2D Semantic Segmentation with Camera-Radar Point Cloud Fusion and Zero-Shot Image Inpaintingcited by 2

Built on by

Not cited yet.