mlsanityData to 14 Oct 2026

Grounded Language-Image Pre-training

arXiv 2112.03857 ↗First posted 7 Dec 2021cs.CV cs.AI cs.CLcited by 1,156

Not on the map. Why?

Cited by 252 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper presents a grounded language-image pre-training (GLIP) model for learning object-level, language-aware, and semantic-rich visual representations. GLIP unifies object detection and phrase grounding for pre-training. The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich. In our experiments, we pre-train GLIP on 27M grounding data, including 3M human-annotated and 24M web-crawled image-text pairs. The learned representations demonstrate strong zero-shot and few-shot transferability to various object-level recognition tasks. 1) When directly evaluated on COCO and LVIS (without seeing any images in COCO during pre-training), GLIP achieves 49.8 AP and 26.9 AP, respectively, surpassing many supervised baselines. 2) After fine-tuned on COCO, GLIP achieves 60.8 AP on val and 61.5 AP on test-dev, surpassing prior SoTA. 3) When transferred to 13 downstream object detection tasks, a 1-shot GLIP rivals with a fully-supervised Dynamic Head. Code is released at https://github.com/microsoft/GLIP.

Key facts. Cited by 1,156 papers in the map (data through 14 Oct 2026). Builds on 59 of its 80 references that are in the map. First posted on arXiv 7 Dec 2021.

Builds on

59 of 80 references.

References · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  9. 2020End-to-End Object Detection with Transformerscited by 7,414
  10. 2017Mask R-CNNcited by 7,277
  11. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  12. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
38 more
  1. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  2. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  5. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  7. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  9. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  10. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  11. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  12. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  13. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  14. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  15. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  16. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  17. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  18. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  19. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  20. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  21. 2017Focal Loss for Dense Object Detectioncited by 752
  22. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  23. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  24. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  25. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  26. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  27. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  28. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  29. 2016R-FCN: Object Detection via Region-based Fully Convolutional Networkscited by 414
  30. 2020Rethinking Pre-training and Self-trainingcited by 352
  31. 2021End-to-End Semi-Supervised Object Detection with Soft Teachercited by 316
  32. 2021Towards Open World Object Detectioncited by 297
  33. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  34. 2020Frustratingly Simple Few-Shot Object Detectioncited by 282
  35. 2018Bilinear Attention Networkscited by 280
  36. 2018Few-shot Object Detection via Feature Reweightingcited by 265
  37. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 250
  38. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 228

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 78; 2023: 303; 2024: 293; 2025: 267; 2026: 214.

1,156 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2025Qwen3-VL Technical Reportcited by 2,793
  4. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  5. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  6. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  7. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  8. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  9. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  10. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  11. 2023ModelScope Text-to-Video Technical Reportcited by 658
  12. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
38 more
  1. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  2. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  3. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  4. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 461
  5. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  6. 2022Multimodal Learning with Transformers: A Surveycited by 400
  7. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  8. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 382
  9. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  10. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  11. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  12. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  13. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  14. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 306
  15. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
  16. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  17. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 300
  18. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
  19. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  20. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  21. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 280
  22. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  23. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  24. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
  25. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  26. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 260
  27. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
  28. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 226
  29. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 198
  30. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
  31. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
  32. 2022Focal Modulation Networkscited by 186
  33. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
  34. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  35. 2022DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detectioncited by 176
  36. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 176
  37. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  38. 2026Qwen3.5-Omni Technical Reportcited by 174

50 most cited of 1,156 All 1,156 (JSON)