Data to 14 Oct 2026

CLIP-Count: Towards Text-Guided Zero-Shot Object Counting

arXiv 2305.07304 ↗First posted 12 May 2023cs.CV cs.AIcited by 83

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Recent advances in visual-language models have shown remarkable zero-shot text-image matching ability that is transferable to downstream tasks such as object detection and segmentation. Adapting these models for object counting, however, remains a formidable challenge. In this study, we first investigate transferring vision-language models (VLMs) for class-agnostic object counting. Specifically, we propose CLIP-Count, the first end-to-end pipeline that estimates density maps for open-vocabulary objects with text guidance in a zero-shot manner. To align the text embedding with dense visual features, we introduce a patch-text contrastive loss that guides the model to learn informative patch-level visual representations for dense prediction. Moreover, we design a hierarchical patch-text interaction module to propagate semantic information across different resolution levels of visual features. Benefiting from the full exploitation of the rich image-text alignment knowledge of pretrained VLMs, our method effectively generates high-quality density maps for objects-of-interest. Extensive experiments on FSC-147, CARPK, and ShanghaiTech crowd counting datasets demonstrate state-of-the-art accuracy and generalizability of the proposed method. Code is available: https://github.com/songrise/CLIP-Count.

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 52 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

41 of 52 references.

Closest ancestors · 3

Same direction, newest first

Other references · 38

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
2018Representation Learning with Contrastive Predictive Codingcited by 7,777
2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
2021Learning to Prompt for Vision-Language Modelscited by 2,281
2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
2021PointCLIP: Point Cloud Understanding by CLIPcited by 463

26 more

2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
2017Drone-based Object Counting by Spatially Regularized Regional Proposal Networkcited by 156
2021Learning To Count Everythingcited by 150
2022NeRF-Art: Text-Driven Neural Radiance Fields Stylizationcited by 126
2023A Closer Look at the Explainability of Contrastive Language-Image Pre-trainingcited by 96
2018Class-Agnostic Countingcited by 85
2022Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Countingcited by 80
2022CounTR: Transformer-based Generalised Visual Countingcited by 79
2022A Low-Shot Object Counting Network With Iterative Prototype Adaptationcited by 65
2022Few-shot Object Counting with Similarity-Aware Feature Enhancementcited by 62
2023Zero-shot Object Countingcited by 57
2017Generating High-Quality Crowd Density Maps using Contextual Pyramid CNNscited by 55
2020Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Countingcited by 52
2019Crowd Counting with Deep Structured Scale Integration Networkcited by 46
2022Learning to Count Anything: Reference-less Class-agnostic Counting with Weak Supervisioncited by 41
2022Exemplar Free Class Agnostic Countingcited by 32
2017FCN-rLSTM: Deep Spatio-Temporal Neural Networks for Vehicle Counting in City Camerascited by 27
2018Crowd Counting using Deep Recurrent Spatial-Aware Networkcited by 21
2022DR.VIC: Decomposition and Reasoning for Video Individual Countingcited by 18
2023GridCLIP: One-Stage Object Detection by Grid-Level CLIP Representation Learningcited by 15
2020Efficient Crowd Counting via Structured Knowledge Transfercited by 13
2021Video Crowd Localization with Multi-focus Gaussian Neighborhood Attention and a Large-Scale Benchmarkcited by 13
2021Object Counting: You Only Need to Look at Onecited by 13
2023GCNet: Probing Self-Similarity Learning for Generalized Counting Networkcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 22; 2025: 30; 2026: 23.

83 papers.

Sort:

By area:

2025AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIPcited by 73
2024CountGD: Multi-Modal Open-World Countingcited by 59
2024DAVE -- A Detect-and-Verify Paradigm for Low-Shot Countingcited by 49
2023VLCounter: Text-aware Visual Representation for Zero-Shot Object Countingcited by 45
2023Point, Segment and Count: A Generalized Framework for Object Countingcited by 39
2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
2023Counting Guidance for High Fidelity Text-to-Image Synthesiscited by 24
2024Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentationcited by 24
2024Zero-shot Object Counting with Good Exemplarscited by 21
2023T-Rex: Counting by Visual Promptingcited by 20
2025T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Countingcited by 19
2025Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Countingcited by 18

38 more

2024OmniCount: Multi-label Object Counting with Semantic-Geometric Priorscited by 16
2024Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answeringcited by 15
2023ABC Easy as 123: A Blind Counter for Exemplar-Free Multi-Class Class-agnostic Countingcited by 13
2024AFreeCA: Annotation-Free Counting for Allcited by 12
2024CLIP-EBC: CLIP Can Count Accurately through Enhanced Blockwise Classificationcited by 12
2024Mind the Prompt: A Novel Benchmark for Prompt-based Class-Agnostic Countingcited by 11
2025Open-World Object Counting in Videoscited by 11
2024OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videoscited by 10
2025Exploring Contextual Attribute Density in Referring Expression Countingcited by 10
2025YOLO-Count: Differentiable Object Counting for Text-to-Image Generationcited by 10
2024Every Shot Counts: Using Exemplars for Repetition Counting in Videoscited by 9
2025CountGD++: Generalized Prompting for Open-World Countingcited by 8
2023X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generationcited by 7
2025A Survey on Class-Agnostic Counting: Advancements from Reference-Based to Open-World Text-Guided Approachescited by 7
2025Generalizing vision-language models to novel domains: A comprehensive surveycited by 7
2025Can Current AI Models Count What We Mean, Not What They See? A Benchmark and Systematic Evaluationcited by 7
2025Single Domain Generalization for Few-Shot Counting via Universal Representation Matchingcited by 6
2025Point-to-Region Loss for Semi-Supervised Point-Based Crowd Countingcited by 6
2025[De|Re]constructing VLMs' Reasoning in Countingcited by 6
2024MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusioncited by 5
2024Robust Zero-Shot Crowd Counting and Localization With Adaptive Resolution SAMcited by 4
2024Detection-Driven Object Count Optimization for Text-to-Image Diffusion Modelscited by 4
2025On the rankability of visual embeddingscited by 4
2023Long-term Wind Power Forecasting with Hierarchical Spatial-Temporal Transformercited by 2
2024TFCounter:Polishing Gems for Training-Free Object Countingcited by 2
2024Ranking-aware adapter for text-driven image ordering with CLIPcited by 2
2025SDVPT: Semantic-Driven Visual Prompt Tuning for Open-World Object Countingcited by 2
2025Text-promptable Object Counting via Quantity Awareness Enhancementcited by 2
2026SITUATE -- Synthetic Object Counting Dataset for VLM trainingcited by 2
2026Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Countingcited by 2
2026Does it Really Count? Assessing Semantic Grounding in Text-Guided Class-Agnostic Countingcited by 2
2024Enhancing Zero-shot Counting via Language-guided Exemplar Learningcited by 1
2024Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmarkcited by 1
2025ProgRoCC: A Progressive Approach to Rough Crowd Countingcited by 1
2025Expanding Zero-Shot Object Counting with Rich Promptscited by 1
2025Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNetcited by 1
2025CountCluster: Training-Free Object Quantity Guidance with Cross-Attention Map Clustering for Text-to-Image Generationcited by 1
2025D2D: Detector-to-Differentiable Critic for Improved Numeracy in Text-to-Image Generationcited by 1

50 most cited of 83 All 83 (JSON)

Similar, not cited

Close in wording, no citation either way