# CLIP-Count: Towards Text-Guided Zero-Shot Object Counting

arXiv: https://arxiv.org/abs/2305.07304 · HTML: https://mlsanity.com/paper/2305.07304/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 41 of its 52 references that are in the map.
- First posted on arXiv 12 May 2023.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 41)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [PointCLIP: Point Cloud Understanding by CLIP](https://mlsanity.com/paper/2112.02413/) (2021)
- [CRIS: CLIP-Driven Referring Image Segmentation](https://mlsanity.com/paper/2111.15174/) (2021)
- [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](https://mlsanity.com/paper/2110.02624/) (2021)
- [Teaching CLIP to Count to Ten](https://mlsanity.com/paper/2302.12066/) (2023)
- [ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2212.03588/) (2022)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [Learning To Count Everything](https://mlsanity.com/paper/2104.08391/) (2021)
- [NeRF-Art: Text-Driven Neural Radiance Fields Stylization](https://mlsanity.com/paper/2212.08070/) (2022)
- [A Closer Look at the Explainability of Contrastive Language-Image Pre-training](https://mlsanity.com/paper/2304.05653/) (2023)
- [Class-Agnostic Counting](https://mlsanity.com/paper/1811.00472/) (2018)
- [Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting](https://mlsanity.com/paper/2203.08354/) (2022)
- [CounTR: Transformer-based Generalised Visual Counting](https://mlsanity.com/paper/2208.13721/) (2022)
- [A Low-Shot Object Counting Network With Iterative Prototype Adaptation](https://mlsanity.com/paper/2211.08217/) (2022)
- [Few-shot Object Counting with Similarity-Aware Feature Enhancement](https://mlsanity.com/paper/2201.08959/) (2022)
- [Zero-shot Object Counting](https://mlsanity.com/paper/2303.02001/) (2023)
- [Generating High-Quality Crowd Density Maps using Contextual Pyramid CNNs](https://mlsanity.com/paper/1708.00953/) (2017)
- [Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting](https://mlsanity.com/paper/2012.04529/) (2020)
- [CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model](https://mlsanity.com/paper/2304.04231/) (2023)

## Built on by (most cited first, 30 of 83)

- [AA-CLIP: Enhancing Zero-shot Anomaly Detection via Anomaly-Aware CLIP](https://mlsanity.com/paper/2503.06661/) (2025)
- [CountGD: Multi-Modal Open-World Counting](https://mlsanity.com/paper/2407.04619/) (2024)
- [DAVE -- A Detect-and-Verify Paradigm for Low-Shot Counting](https://mlsanity.com/paper/2404.16622/) (2024)
- [VLCounter: Text-aware Visual Representation for Zero-Shot Object Counting](https://mlsanity.com/paper/2312.16580/) (2023)
- [Point, Segment and Count: A Generalized Framework for Object Counting](https://mlsanity.com/paper/2311.12386/) (2023)
- [R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding](https://mlsanity.com/paper/2404.00801/) (2024)
- [Counting Guidance for High Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2306.17567/) (2023)
- [Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2411.17150/) (2024)
- [Zero-shot Object Counting with Good Exemplars](https://mlsanity.com/paper/2407.04948/) (2024)
- [T-Rex: Counting by Visual Prompting](https://mlsanity.com/paper/2311.13596/) (2023)
- [T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting](https://mlsanity.com/paper/2502.20625/) (2025)
- [Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting](https://mlsanity.com/paper/2510.04401/) (2025)
- [OmniCount: Multi-label Object Counting with Semantic-Geometric Priors](https://mlsanity.com/paper/2403.05435/) (2024)
- [Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering](https://mlsanity.com/paper/2403.14783/) (2024)
- [ABC Easy as 123: A Blind Counter for Exemplar-Free Multi-Class Class-agnostic Counting](https://mlsanity.com/paper/2309.04820/) (2023)
- [AFreeCA: Annotation-Free Counting for All](https://mlsanity.com/paper/2403.04943/) (2024)
- [CLIP-EBC: CLIP Can Count Accurately through Enhanced Blockwise Classification](https://mlsanity.com/paper/2403.09281/) (2024)
- [Mind the Prompt: A Novel Benchmark for Prompt-based Class-Agnostic Counting](https://mlsanity.com/paper/2409.15953/) (2024)
- [Open-World Object Counting in Videos](https://mlsanity.com/paper/2506.15368/) (2025)
- [OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videos](https://mlsanity.com/paper/2407.17085/) (2024)
- [Exploring Contextual Attribute Density in Referring Expression Counting](https://mlsanity.com/paper/2503.12460/) (2025)
- [YOLO-Count: Differentiable Object Counting for Text-to-Image Generation](https://mlsanity.com/paper/2508.00728/) (2025)
- [Every Shot Counts: Using Exemplars for Repetition Counting in Videos](https://mlsanity.com/paper/2403.18074/) (2024)
- [CountGD++: Generalized Prompting for Open-World Counting](https://mlsanity.com/paper/2512.23351/) (2025)
- [X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation](https://mlsanity.com/paper/2312.00085/) (2023)
- [A Survey on Class-Agnostic Counting: Advancements from Reference-Based to Open-World Text-Guided Approaches](https://mlsanity.com/paper/2501.19184/) (2025)
- [Generalizing vision-language models to novel domains: A comprehensive survey](https://mlsanity.com/paper/2506.18504/) (2025)
- [Can Current AI Models Count What We Mean, Not What They See? A Benchmark and Systematic Evaluation](https://mlsanity.com/paper/2509.13939/) (2025)
- [Single Domain Generalization for Few-Shot Counting via Universal Representation Matching](https://mlsanity.com/paper/2505.16778/) (2025)
- [Point-to-Region Loss for Semi-Supervised Point-Based Crowd Counting](https://mlsanity.com/paper/2505.21943/) (2025)

## Abstract (arXiv)

Recent advances in visual-language models have shown remarkable zero-shot text-image matching ability that is transferable to downstream tasks such as object detection and segmentation. Adapting these models for object counting, however, remains a formidable challenge. In this study, we first investigate transferring vision-language models (VLMs) for class-agnostic object counting. Specifically, we propose CLIP-Count, the first end-to-end pipeline that estimates density maps for open-vocabulary objects with text guidance in a zero-shot manner. To align the text embedding with dense visual features, we introduce a patch-text contrastive loss that guides the model to learn informative patch-level visual representations for dense prediction. Moreover, we design a hierarchical patch-text interaction module to propagate semantic information across different resolution levels of visual features. Benefiting from the full exploitation of the rich image-text alignment knowledge of pretrained VLMs, our method effectively generates high-quality density maps for objects-of-interest. Extensive experiments on FSC-147, CARPK, and ShanghaiTech crowd counting datasets demonstrate state-of-the-art accuracy and generalizability of the proposed method. Code is available: https://github.com/songrise/CLIP-Count.