# An Open and Comprehensive Pipeline for Unified Object Grounding and Detection

arXiv: https://arxiv.org/abs/2401.02361 · HTML: https://mlsanity.com/paper/2401.02361/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 4 Jan 2024.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models](https://mlsanity.com/paper/2204.08790/) (2022)
- [Described Object Detection: Liberating Object Detection with Flexible Expressions](https://mlsanity.com/paper/2307.12813/) (2023)
- [Cascade-DETR: Delving into High-Quality Universal Object Detection](https://mlsanity.com/paper/2307.11035/) (2023)

## Built on by (most cited first, 30 of 65)

- [EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain](https://mlsanity.com/paper/2401.16822/) (2024)
- [Multimodal Autoregressive Pre-training of Large Vision Encoders](https://mlsanity.com/paper/2411.14402/) (2024)
- [DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding](https://mlsanity.com/paper/2411.14347/) (2024)
- [LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models](https://mlsanity.com/paper/2501.18954/) (2025)
- [OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference](https://mlsanity.com/paper/2502.18411/) (2025)
- [Learning Visual Grounding from Generative Vision and Language Model](https://mlsanity.com/paper/2407.14563/) (2024)
- [DynamicEarth: How Far are We from Open-Vocabulary Change Detection?](https://mlsanity.com/paper/2501.12931/) (2025)
- [FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension](https://mlsanity.com/paper/2409.14750/) (2024)
- [3D-GRES: Generalized 3D Referring Expression Segmentation](https://mlsanity.com/paper/2407.20664/) (2024)
- [MMRo: Are Multimodal LLMs Eligible as the Brain for In-Home Robotics?](https://mlsanity.com/paper/2406.19693/) (2024)
- [Popeye: A Unified Visual-Language Model for Multi-Source Ship Detection from Remote Sensing Imagery](https://mlsanity.com/paper/2403.03790/) (2024)
- [WeDetect: Fast Open-Vocabulary Object Detection as Retrieval](https://mlsanity.com/paper/2512.12309/) (2025)
- [ToolVQA: A Dataset for Multi-step Reasoning VQA with External Tools](https://mlsanity.com/paper/2508.03284/) (2025)
- [FG-CLIP 2: A Bilingual Fine-grained Vision-Language Alignment Model](https://mlsanity.com/paper/2510.10921/) (2025)
- [The Second Challenge on Cross-Domain Few-Shot Object Detection at NTIRE 2026: Methods and Results](https://mlsanity.com/paper/2604.11998/) (2026)
- [3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection](https://mlsanity.com/paper/2507.23567/) (2025)
- [Grounding Partially-Defined Events in Multimodal Data](https://mlsanity.com/paper/2410.05267/) (2024)
- [New Dataset and Methods for Fine-Grained Compositional Referring Expression Comprehension via Specialist-MLLM Collaboration](https://mlsanity.com/paper/2502.20104/) (2025)
- [DINO-R1: Incentivizing Reasoning Capability in Vision Foundation Models](https://mlsanity.com/paper/2505.24025/) (2025)
- [Referring Expression Comprehension for Small Objects](https://mlsanity.com/paper/2510.03701/) (2025)
- [GREx: Generalized Referring Expression Segmentation, Comprehension, and Generation](https://mlsanity.com/paper/2601.05244/) (2026)
- [STRIVE: Structured Representation Integrating VLM Reasoning for Efficient Object Navigation](https://mlsanity.com/paper/2505.06729/) (2025)
- [CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning](https://mlsanity.com/paper/2510.06243/) (2025)
- [Auto Cherry-Picker: Learning from High-quality Generative Data Driven by Language](https://mlsanity.com/paper/2406.20085/) (2024)
- [Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Tracking](https://mlsanity.com/paper/2503.16538/) (2025)
- [Re-Aligning Language to Visual Objects with an Agentic Workflow](https://mlsanity.com/paper/2503.23508/) (2025)
- [VLA^2: Empowering Vision-Language-Action Models with an Agentic Framework for Unseen Concept Manipulation](https://mlsanity.com/paper/2510.14902/) (2025)
- [Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary Detection](https://mlsanity.com/paper/2409.08513/) (2024)
- [PET-DINO: Unifying Visual Cues into Grounding DINO with Prompt-Enriched Training](https://mlsanity.com/paper/2604.00503/) (2026)
- [DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images](https://mlsanity.com/paper/2607.00338/) (2026)

## Abstract (arXiv)

Grounding-DINO is a state-of-the-art open-set detection model that tackles multiple vision tasks including Open-Vocabulary Detection (OVD), Phrase Grounding (PG), and Referring Expression Comprehension (REC). Its effectiveness has led to its widespread adoption as a mainstream architecture for various downstream applications. However, despite its significance, the original Grounding-DINO model lacks comprehensive public technical details due to the unavailability of its training code. To bridge this gap, we present MM-Grounding-DINO, an open-source, comprehensive, and user-friendly baseline, which is built with the MMDetection toolbox. It adopts abundant vision datasets for pre-training and various detection and grounding datasets for fine-tuning. We give a comprehensive analysis of each reported result and detailed settings for reproduction. The extensive experiments on the benchmarks mentioned demonstrate that our MM-Grounding-DINO-Tiny outperforms the Grounding-DINO-Tiny baseline. We release all our models to the research community. Codes and trained models are released at https://github.com/open-mmlab/mmdetection/tree/main/configs/mm_grounding_dino.