mlsanityData to 7 Oct 2026

WaterVG: Waterway Visual Grounding based on Text-Guided Vision and mmWave Radar

arXiv 2403.12686 ↗First posted 19 Mar 2024cs.CV cs.MM cs.ROcited by 13

Not on the map. Why?

Abstract (arXiv)

The perception of waterways based on human intent is significant for autonomous navigation and operations of Unmanned Surface Vehicles (USVs) in water environments. Inspired by visual grounding, we introduce WaterVG, the first visual grounding dataset designed for USV-based waterway perception based on human prompts. WaterVG encompasses prompts describing multiple targets, with annotations at the instance level including bounding boxes and masks. Notably, WaterVG includes 11,568 samples with 34,987 referred targets, whose prompts integrates both visual and radar characteristics. The pattern of text-guided two sensors equips a finer granularity of text prompts with visual and radar features of referred targets. Moreover, we propose a low-power visual grounding model, Potamoi, which is a multi-task model with a well-designed Phased Heterogeneous Modality Fusion (PHMF) mode, including Adaptive Radar Weighting (ARW) and Multi-Head Slim Cross Attention (MHSCA). Exactly, ARW extracts required radar features to fuse with vision for prompt alignment. MHSCA is an efficient fusion module with a remarkably small parameter count and FLOPs, elegantly fusing scenario context captured by two sensors with linguistic features, which performs expressively on visual grounding tasks. Comprehensive experiments and evaluations have been conducted on WaterVG, where our Potamoi archives state-of-the-art performances compared with counterparts.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 62 of its 103 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

62 of 103 references.

References · 62

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2015Fast R-CNNcited by 2,930
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  10. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  11. 2020Rethinking Attention with Performerscited by 1,584
  12. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
38 more
  1. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  2. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  3. 2016Modeling Context in Referring Expressionscited by 1,054
  4. 2017Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentationscited by 614
  5. 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 454
  6. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  7. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 408
  8. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  9. 2022YOLOv6: A Single-Stage Object Detection Framework for Industrial Applicationscited by 273
  10. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  11. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  12. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  13. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 194
  14. 2023FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterizationcited by 161
  15. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  16. 2019Talk2Car: Taking Control of Your Self-Driving Carcited by 141
  17. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 139
  18. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  19. 2022Multimodal Token Fusion for Vision Transformerscited by 132
  20. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  21. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 115
  22. 2019Dynamic Graph Attention for Referring Expression Comprehensioncited by 111
  23. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  24. 2018Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networkscited by 102
  25. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  26. 2023Referring Multi-Object Trackingcited by 91
  27. 2020Referring Expression Comprehension: A Survey of Methods and Datasetscited by 78
  28. 2021SeaDronesSee: A Maritime Benchmark for Detecting Humans in Open Watercited by 59
  29. 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
  30. 2018Real-Time Referring Expression Comprehension by Single-Stage Grounding Networkcited by 56
  31. 2021SA-Net: Shuffle Attention for Deep Convolutional Neural Networkscited by 54
  32. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  33. 2020Cops-Ref: A new Dataset and Task on Compositional Referring Expression Comprehensioncited by 50
  34. 2021Visual Grounding with Transformerscited by 42
  35. 2023WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfacescited by 34
  36. 2022PP-LiteSeg: A Superior Real-Time Semantic Segmentation Modelcited by 33
  37. 2021Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniquescited by 32
  38. 2021MODS -- A USV-oriented object detection and obstacle segmentation benchmarkcited by 27

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 2.

13 papers.

Sort:

By area:

  1. 2025Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perceptioncited by 21
  2. 2024Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehensioncited by 17
  3. 2024NanoMVG: USV-Centric Low-Power Multi-Task Visual Grounding based on Prompt-Guided Camera and 4D mmWave Radarcited by 7
  4. 2025Cognitive Disentanglement for Referring Multi-Object Trackingcited by 6
  5. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  6. 2025MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategiescited by 3
  7. 2025RoadSceneVQA: Benchmarking Visual Question Answering in Roadside Perception Systems for Intelligent Transportation Systemcited by 3
  8. 2025AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarioscited by 3
  9. 2024Referring Flexible Image Restorationcited by 0
  10. 2025Talk2PC: Enhancing 3D Visual Grounding through LiDAR and Radar Point Clouds Fusion for Autonomous Drivingcited by 0
  11. 2025Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understandingcited by 0
  12. 2026WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agentscited by 0
1 more
  1. 2026RCVLA: 4D Radar-Grounded Semantic Reasoning and Trajectory Arbitration for Autonomous Drivingcited by 0