mlsanityData to 13 Oct 2026

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

arXiv 2303.05499 ↗First posted 9 Mar 2023cs.CVcited by 3,299

Not on the map. Why?

Cited by 580 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we present an open-set object detector, called Grounding DINO, by marrying Transformer-based detector DINO with grounded pre-training, which can detect arbitrary objects with human inputs such as category names or referring expressions. The key solution of open-set object detection is introducing language to a closed-set detector for open-set concept generalization. To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion. While previous works mainly evaluate open-set object detection on novel categories, we propose to also perform evaluations on referring expression comprehension for objects specified with attributes. Grounding DINO performs remarkably well on all three settings, including benchmarks on COCO, LVIS, ODinW, and RefCOCO/+/g. Grounding DINO achieves a 52.5 AP on the COCO detection zero-shot transfer benchmark, i.e., without any training data from COCO. It sets a new record on the ODinW zero-shot benchmark with a mean 26.1 AP. Code will be available at urlhttps://github.com/IDEA-Research/GroundingDINO.

Key facts. Cited by 3,299 papers in the map (data through 13 Oct 2026). Builds on 53 of its 69 references that are in the map. First posted on arXiv 9 Mar 2023.

Builds on

53 of 69 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,111
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,625
  7. 2020End-to-End Object Detection with Transformerscited by 7,396
  8. 2017Mask R-CNNcited by 7,266
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,417
  11. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,910
  12. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,649
38 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,215
  2. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,362
  3. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,355
  4. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,312
  5. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  6. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  7. 2019Objects as Pointscited by 1,173
  8. 2021Grounded Language-Image Pre-trainingcited by 1,152
  9. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,023
  10. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  11. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 815
  12. 2021Florence: A New Foundation Model for Computer Visioncited by 803
  13. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 777
  14. 2017Focal Loss for Dense Object Detectioncited by 750
  15. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 615
  16. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  17. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 571
  18. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 539
  19. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  20. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  21. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  22. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 430
  23. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  24. 2021Conditional DETR for Fast Training Convergencecited by 423
  25. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 343
  26. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  27. 2021End-to-End Semi-Supervised Object Detection with Soft Teachercited by 315
  28. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 304
  29. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  30. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  31. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  32. 2021Dynamic Head: Unifying Object Detection Heads with Attentionscited by 186
  33. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  34. 2022DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detectioncited by 176
  35. 2022DETRs with Hybrid Matchingcited by 150
  36. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  37. 2022ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Modelscited by 141
  38. 2022Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignmentcited by 97

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 308; 2024: 826; 2025: 1186; 2026: 979.

3,299 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,827
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,191
  3. 2025Qwen3-VL Technical Reportcited by 2,663
  4. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,610
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,798
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,452
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,353
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,082
  9. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,002
  10. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 919
  11. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 898
  12. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 773
38 more
  1. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 706
  2. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 621
  3. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 520
  4. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 506
  5. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 500
  6. 2025VACE: All-in-One Video Creation and Editingcited by 473
  7. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 470
  8. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 469
  9. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 448
  10. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 406
  11. 2024OmniGen: Unified Image Generationcited by 392
  12. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 389
  13. 2023Segment Anything in High Qualitycited by 374
  14. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 373
  15. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 359
  16. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 347
  17. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 343
  18. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  19. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 322
  20. 2023Recognize Anything: A Strong Image Tagging Modelcited by 306
  21. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 303
  22. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 300
  23. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 299
  24. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  25. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 288
  26. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 285
  27. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 273
  28. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 267
  29. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 263
  30. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 255
  31. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 249
  32. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 247
  33. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  34. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 237
  35. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 226
  36. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  37. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 218
  38. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 218

50 most cited of 3,299 All 3,299 (JSON)