# CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detection

arXiv: https://arxiv.org/abs/2310.16667 · HTML: https://mlsanity.com/paper/2310.16667/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 64 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 30 of 52)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Open-Vocabulary Object Detection Using Captions](https://mlsanity.com/paper/2011.10678/) (2020)
- [EVA-02: A Visual Representation for Neon Genesis](https://mlsanity.com/paper/2303.11331/) (2023)
- [Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model](https://mlsanity.com/paper/2203.14940/) (2022)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [Open-Vocabulary DETR with Conditional Matching](https://mlsanity.com/paper/2203.11876/) (2022)
- [Universal Instance Perception as Object Discovery and Retrieval](https://mlsanity.com/paper/2303.06674/) (2023)
- [Zero-Shot Object Detection](https://mlsanity.com/paper/1804.04340/) (2018)

## Built on by (most cited first, 30 of 46)

- [Detect Anything via Next Point Prediction](https://mlsanity.com/paper/2510.12798/) (2025)
- [General Object Foundation Model for Images and Videos at Scale](https://mlsanity.com/paper/2312.09158/) (2023)
- [AffordanceLLM: Grounding Affordance from Vision Language Models](https://mlsanity.com/paper/2401.06341/) (2024)
- [DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception](https://mlsanity.com/paper/2505.04410/) (2025)
- [Generative Region-Language Pretraining for Open-Ended Object Detection](https://mlsanity.com/paper/2403.10191/) (2024)
- [OV-DQUO: Open-Vocabulary DETR with Denoising Text Query Training and Open-World Unknown Objects Supervision](https://mlsanity.com/paper/2405.17913/) (2024)
- [Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments](https://mlsanity.com/paper/2403.13556/) (2024)
- [LaMI-DETR: Open-Vocabulary Detection with Language Model Instruction](https://mlsanity.com/paper/2407.11335/) (2024)
- [PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects](https://mlsanity.com/paper/2407.16696/) (2024)
- [UMG-CLIP: A Unified Multi-Granularity Vision Generalist for Open-World Understanding](https://mlsanity.com/paper/2401.06397/) (2024)
- [Refining CLIP's Spatial Awareness: A Visual-Centric Perspective](https://mlsanity.com/paper/2504.02328/) (2025)
- [As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?](https://mlsanity.com/paper/2403.12693/) (2024)
- [SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detection](https://mlsanity.com/paper/2405.10053/) (2024)
- [Boosting Segment Anything Model Towards Open-Vocabulary Learning](https://mlsanity.com/paper/2312.03628/) (2023)
- [Vision-Language Model for Object Detection and Segmentation: A Review and Evaluation](https://mlsanity.com/paper/2504.09480/) (2025)
- [What does CLIP know about peeling a banana?](https://mlsanity.com/paper/2404.12015/) (2024)
- [OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection](https://mlsanity.com/paper/2411.17761/) (2024)
- [YOLO-UniOW: Efficient Universal Open-World Object Detection](https://mlsanity.com/paper/2412.20645/) (2024)
- [A Hierarchical Semantic Distillation Framework for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2503.10152/) (2025)
- [Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2503.11005/) (2025)
- [Decoupled Global-Local Alignment for Improving Compositional Understanding](https://mlsanity.com/paper/2504.16801/) (2025)
- [Vocabulary-free Image Classification and Semantic Segmentation](https://mlsanity.com/paper/2404.10864/) (2024)
- [Dynamic Object Queries for Transformer-based Incremental Object Detection](https://mlsanity.com/paper/2407.21687/) (2024)
- [Recognize Any Regions](https://mlsanity.com/paper/2311.01373/) (2023)
- [Language-conditioned Detection Transformer](https://mlsanity.com/paper/2311.17902/) (2023)
- [OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking](https://mlsanity.com/paper/2410.17534/) (2024)
- [C3-OWD: A Curriculum Cross-modal Contrastive Learning Framework for Open-World Detection](https://mlsanity.com/paper/2509.23316/) (2025)
- [Training-free Boost for Open-Vocabulary Object Detection with Confidence Aggregation](https://mlsanity.com/paper/2404.08603/) (2024)
- [RTGen: Generating Region-Text Pairs for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2405.19854/) (2024)
- [OVMR: Open-Vocabulary Recognition with Multi-Modal References](https://mlsanity.com/paper/2406.04675/) (2024)

## Abstract (arXiv)

Deriving reliable region-word alignment from image-text pairs is critical to learn object-level vision-language representations for open-vocabulary object detection. Existing methods typically rely on pre-trained or self-trained vision-language models for alignment, which are prone to limitations in localization accuracy or generalization capabilities. In this paper, we propose CoDet, a novel approach that overcomes the reliance on pre-aligned vision-language space by reformulating region-word alignment as a co-occurring object discovery problem. Intuitively, by grouping images that mention a shared concept in their captions, objects corresponding to the shared concept shall exhibit high co-occurrence among the group. CoDet then leverages visual similarities to discover the co-occurring objects and align them with the shared concept. Extensive experiments demonstrate that CoDet has superior performances and compelling scalability in open-vocabulary detection, e.g., by scaling up the visual backbone, CoDet achieves 37.0 APmnovel and 44.7 APmall on OV-LVIS, surpassing the previous SoTA by 4.2 APmnovel and 9.8 APmall. Code is available at https://github.com/CVMI-Lab/CoDet.