mlsanityData to 14 Oct 2026

YOLO-World: Real-Time Open-Vocabulary Object Detection

arXiv 2401.17270 ↗First posted 30 Jan 2024cs.CVcited by 503

Not on the map. Why?

Cited by 76 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The You Only Look Once (YOLO) series of detectors have established themselves as efficient and practical tools. However, their reliance on predefined and trained object categories limits their applicability in open scenarios. Addressing this limitation, we introduce YOLO-World, an innovative approach that enhances YOLO with open-vocabulary detection capabilities through vision-language modeling and pre-training on large-scale datasets. Specifically, we propose a new Re-parameterizable Vision-Language Path Aggregation Network (RepVL-PAN) and region-text contrastive loss to facilitate the interaction between visual and linguistic information. Our method excels in detecting a wide range of objects in a zero-shot manner with high efficiency. On the challenging LVIS dataset, YOLO-World achieves 35.4 AP with 52.0 FPS on V100, which outperforms many state-of-the-art methods in terms of both accuracy and speed. Furthermore, the fine-tuned YOLO-World achieves remarkable performance on several downstream tasks, including object detection and open-vocabulary instance segmentation.

Key facts. Cited by 503 papers in the map (data through 14 Oct 2026). Builds on 53 of its 69 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

53 of 69 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  9. 2020End-to-End Object Detection with Transformerscited by 7,414
  10. 2017Mask R-CNNcited by 7,277
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  12. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
38 more
  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  2. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  4. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  5. 2015Fast R-CNNcited by 2,933
  6. 2018YOLOv3: An Incremental Improvementcited by 2,659
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  9. 2016YOLO9000: Better, Faster, Strongercited by 1,431
  10. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  11. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  12. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  13. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  14. 2021Grounded Language-Image Pre-trainingcited by 1,156
  15. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,071
  16. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 982
  17. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  18. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  19. 2018Path Aggregation Network for Instance Segmentationcited by 798
  20. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  21. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  22. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 540
  23. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
  24. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  25. 2019CSPNet: A New Backbone that can Enhance Learning Capability of CNNcited by 381
  26. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  27. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  28. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
  29. 2022YOLOv6: A Single-Stage Object Detection Framework for Industrial Applicationscited by 275
  30. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  31. 2021TOOD: Task-aligned One-stage Object Detectioncited by 218
  32. 2022DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detectioncited by 176
  33. 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 150
  34. 2023Aligning Bag of Regions for Open-Vocabulary Object Detectioncited by 119
  35. 2022Learning Object-Language Alignments for Open-Vocabulary Object Detectioncited by 96
  36. 2023DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignmentcited by 90
  37. 2021Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labelingcited by 84
  38. 2022PP-YOLOE: An evolved version of YOLOcited by 75

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 103; 2025: 214; 2026: 186.

503 papers.

Sort:

By area:

  1. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  2. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  3. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
  4. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 152
  5. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  6. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 89
  7. 2025FG-CLIP: Fine-Grained Visual and Textual Alignmentcited by 81
  8. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 75
  9. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  10. 2025Detect Anything via Next Point Predictioncited by 73
  11. 2025UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generationcited by 66
  12. 2025LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Modelscited by 57
38 more
  1. 2026Ultralytics YOLO26: Unified Real-Time End-to-End Vision Modelscited by 57
  2. 2024VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Groundingcited by 56
  3. 2025VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learningcited by 51
  4. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 49
  5. 2025World-in-World: World Models in a Closed-Loop Worldcited by 47
  6. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  7. 20253DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Modelcited by 37
  8. 2025YOLOE: Real-Time Seeing Anythingcited by 36
  9. 2025Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understandingcited by 34
  10. 2024Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentationcited by 30
  11. 2025ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundationscited by 30
  12. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  13. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  14. 2024OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusioncited by 28
  15. 2025IMAGHarmony: Controllable Image Editing with Consistent Object Quantity and Layoutcited by 28
  16. 2025Mitigating Object Hallucinations via Sentence-Level Early Interventioncited by 28
  17. 2024WorldScribe: Towards Context-Aware Live Visual Descriptionscited by 27
  18. 2024RAP: Retrieval-Augmented Personalization for Multimodal Large Language Modelscited by 24
  19. 2024Embodied VideoAgent: Persistent Memory from Egocentric Videos and Embodied Sensors Enables Dynamic Scene Understandingcited by 24
  20. 2025PUGS: Zero-shot Physical Understanding with Gaussian Splattingcited by 24
  21. 2024REvolve: Reward Evolution with Large Language Models using Human Feedbackcited by 23
  22. 2024Open Vocabulary Monocular 3D Object Detectioncited by 23
  23. 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interactioncited by 23
  24. 2024RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-Worldcited by 22
  25. 2024ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Modelscited by 21
  26. 2025Referring to Any Personcited by 21
  27. 2024Real-time Transformer-based Open-Vocabulary Detection with Efficient Fusion Headcited by 20
  28. 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigationcited by 20
  29. 2024Interpreting Object-level Foundation Models via Visual Precision Searchcited by 20
  30. 2024Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentationcited by 20
  31. 2025TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learningcited by 20
  32. 2025GraphCoT-VLA: A 3D Spatial-Aware Reasoning Vision-Language-Action Model for Robotic Manipulation with Ambiguous Instructionscited by 19
  33. 2025PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineercited by 19
  34. 2024Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentationcited by 17
  35. 2024RGBT Tracking via All-layer Multimodal Interactions with Progressive Fusion Mambacited by 17
  36. 2024Language Driven Occupancy Predictioncited by 17
  37. 2024Functionality understanding and segmentation in 3D scenescited by 17
  38. 2024The 8th AI City Challengecited by 16

50 most cited of 503 All 503 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Tracking
  2. 2025Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models
  3. 2023DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detection
  4. 2023Towards Open Vocabulary Learning: A Survey
  5. 2024Comprehensive Multi-Modal Prototypes are Simple and Effective Classifiers for Vast-Vocabulary Object Detection
  6. 2024LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained Descriptors
  7. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Future
  8. 2021Open Vocabulary Object Detection with Pseudo Bounding-Box Labels