mlsanityData to 14 Oct 2026

RefDrone: A Challenging Benchmark for Referring Expression Comprehension in Drone Scenes

arXiv 2502.00392 ↗First posted 1 Feb 2025cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Drones have become prevalent robotic platforms with diverse applications, showing significant potential in Embodied Artificial Intelligence (Embodied AI). Referring Expression Comprehension (REC) enables drones to locate objects based on natural language expressions, a crucial capability for Embodied AI. Despite advances in REC for ground-level scenes, aerial views introduce unique challenges including varying viewpoints, occlusions and scale variations. To address this gap, we introduce RefDrone, a REC benchmark for drone scenes. RefDrone reveals three key challenges in REC: 1) multi-scale and small-scale target detection; 2) multi-target and no-target samples; 3) complex environment with rich contextual expressions. To efficiently construct this dataset, we develop RDAgent (referring drone annotation framework with multi-agent system), a semi-automated annotation tool for REC tasks. RDAgent ensures high-quality contextual expressions and reduces annotation cost. Furthermore, we propose Number GroundingDINO (NGDINO), a novel method designed to handle multi-target and no-target cases. NGDINO explicitly learns and utilizes the number of objects referred to in the expression. Comprehensive experiments with state-of-the-art REC methods demonstrate that NGDINO achieves superior performance on both the proposed RefDrone and the existing gRefCOCO datasets. The dataset and code are be publicly at https://github.com/sunzc-sunny/refdrone.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 58 of its 74 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

58 of 74 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2016Wide Residual Networkscited by 3,070
  8. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  9. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  10. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  11. 2021Grounded Language-Image Pre-trainingcited by 1,156
  12. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
38 more
  1. 2016Modeling Context in Referring Expressionscited by 1,063
  2. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  3. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  4. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  5. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  6. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  7. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  8. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  9. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  10. 2019Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmarkcited by 340
  11. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  12. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  13. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
  14. 2020Detection and Tracking Meet Drones Challengecited by 263
  15. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  16. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
  17. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 184
  18. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  19. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 146
  20. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 140
  21. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 123
  22. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  23. 2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehensioncited by 120
  24. 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 118
  25. 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 113
  26. 2016Modeling Relationships in Referential Expressions with Compositional Modular Networkscited by 106
  27. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  28. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 89
  29. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 77
  30. 2019CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressionscited by 73
  31. 2021SeaDronesSee: A Maritime Benchmark for Detecting Humans in Open Watercited by 61
  32. 2020Cops-Ref: A new Dataset and Task on Compositional Referring Expression Comprehensioncited by 50
  33. 2022Aerial Vision-and-Dialog Navigationcited by 49
  34. 2024COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Modelscited by 45
  35. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  36. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  37. 2021YouRefIt: Embodied Reference Understanding with Language and Gesturecited by 33
  38. 2023Griffon: Spelling out All Object Locations at Any Granularity with Large Language Modelscited by 31

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 14.

19 papers.

Sort:

By area:

  1. 2025Multimodal Referring Segmentation: A Surveycited by 22
  2. 2025MM-UAVBench: How Well Do Multimodal Large Language Models See, Think, and Plan in Low-Altitude UAV Scenarios?cited by 10
  3. 2026APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigationcited by 5
  4. 2026UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understandingcited by 5
  5. 2025AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarioscited by 4
  6. 2026UAV-Track VLA: Embodied Aerial Tracking via Vision-Language-Action Modelscited by 4
  7. 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescuecited by 4
  8. 2025RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentationcited by 1
  9. 2025Comprehension of Multilingual Expressions Referring to Target Objects in Visual Inputscited by 1
  10. 2026AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Groundingcited by 1
  11. 2026CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Trackingcited by 1
  12. 2026RSVideo: Are Your Vision-Language Models Ready for Remote Sensing Videos?cited by 1
7 more
  1. 2026Robust Grounding with MLLMs Against Occlusion and Small Objects via Language-Guided Semantic Cuescited by 0
  2. 2026Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checkercited by 0
  3. 2026A-PAIR: A Benchmark and Identity-Consistent Grounding Framework for Air-Ground Cross-View Referring Person Detectioncited by 0
  4. 2026VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Modelscited by 0
  5. 2026Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environmentscited by 0
  6. 2026Knowing Whether, Not Just Where: Anchor-Probing Distillation for Open-Set Remote Sensing Visual Groundingcited by 0
  7. 2026The Gap Is in the Geometry: A Convergent Meta-Analysis of Multi-Facet Aerial and Remote Sensing Video Understandingcited by 0