# NanoMVG: USV-Centric Low-Power Multi-Task Visual Grounding based on Prompt-Guided Camera and 4D mmWave Radar

arXiv: https://arxiv.org/abs/2408.17207 · HTML: https://mlsanity.com/paper/2408.17207/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 54 references that are in the map.
- First posted on arXiv 30 Aug 2024.

## Builds on (most cited first, 30 of 41)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://mlsanity.com/paper/2004.02984/) (2020)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [TransVG: End-to-End Visual Grounding with Transformers](https://mlsanity.com/paper/2104.08541/) (2021)
- [Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation](https://mlsanity.com/paper/2003.08813/) (2020)
- [A Fast and Accurate One-Stage Approach to Visual Grounding](https://mlsanity.com/paper/1908.06354/) (2019)
- [Referring Transformer: A One-step Approach to Multi-task Visual Grounding](https://mlsanity.com/paper/2106.03089/) (2021)
- [SeqTR: A Simple yet Universal Network for Visual Grounding](https://mlsanity.com/paper/2203.16265/) (2022)
- [Dynamic Graph Attention for Referring Expression Comprehension](https://mlsanity.com/paper/1909.08164/) (2019)
- [VLT: Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2210.15871/) (2022)
- [MobileOne: An Improved One millisecond Mobile Backbone](https://mlsanity.com/paper/2206.04040/) (2022)
- [Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networks](https://mlsanity.com/paper/1812.04794/) (2018)
- [MobileNetV4 -- Universal Models for the Mobile Ecosystem](https://mlsanity.com/paper/2404.10518/) (2024)
- [Learning to Compose and Reason with Language Tree Structures for Visual Grounding](https://mlsanity.com/paper/1906.01784/) (2019)
- [Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2005.03572/) (2020)
- [Referring Expression Comprehension: A Survey of Methods and Datasets](https://mlsanity.com/paper/2007.09554/) (2020)
- [Multi-Modal 3D Object Detection in Autonomous Driving: a Survey](https://mlsanity.com/paper/2106.12735/) (2021)
- [SeaDronesSee: A Maritime Benchmark for Detecting Humans in Open Water](https://mlsanity.com/paper/2105.01922/) (2021)
- [LXL: LiDAR Excluded Lean 3D Object Detection with 4D Imaging Radar and Camera Fusion](https://mlsanity.com/paper/2307.00724/) (2023)
- [Real-Time Referring Expression Comprehension by Single-Stage Grounding Network](https://mlsanity.com/paper/1812.03426/) (2018)
- [Contrastive Grouping with Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2309.01017/) (2023)
- [SMURF: Spatial Multi-Representation Fusion for 3D Object Detection with 4D Imaging Radar](https://mlsanity.com/paper/2307.10784/) (2023)
- [WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfaces](https://mlsanity.com/paper/2307.06505/) (2023)
- [Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation](https://mlsanity.com/paper/1802.07956/) (2018)

## Built on by (most cited first, 7 of 7)

- [Doracamom: Joint 3D Detection and Occupancy Prediction with Multi-view 4D Radars and Cameras for Omnidirectional Perception](https://mlsanity.com/paper/2501.15394/) (2025)
- [AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarios](https://mlsanity.com/paper/2511.21053/) (2025)
- [MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategies](https://mlsanity.com/paper/2501.15384/) (2025)
- [RadarNeXt: Real-Time and Reliable 3D Object Detector Based On 4D mmWave Imaging Radar](https://mlsanity.com/paper/2501.02314/) (2025)
- [Da Yu: Towards USV-Based Image Captioning for Waterway Surveillance and Scene Understanding](https://mlsanity.com/paper/2506.19288/) (2025)
- [WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agents](https://mlsanity.com/paper/2602.22923/) (2026)
- [RCVLA: 4D Radar-Grounded Semantic Reasoning and Trajectory Arbitration for Autonomous Driving](https://mlsanity.com/paper/2609.32681/) (2026)

## Abstract (arXiv)

Recently, visual grounding and multi-sensors setting have been incorporated into perception system for terrestrial autonomous driving systems and Unmanned Surface Vehicles (USVs), yet the high complexity of modern learning-based visual grounding model using multi-sensors prevents such model to be deployed on USVs in the real-life. To this end, we design a low-power multi-task model named NanoMVG for waterway embodied perception, guiding both camera and 4D millimeter-wave radar to locate specific object(s) through natural language. NanoMVG can perform both box-level and mask-level visual grounding tasks simultaneously. Compared to other visual grounding models, NanoMVG achieves highly competitive performance on the WaterVG dataset, particularly in harsh environments and boasts ultra-low power consumption for long endurance.