# VLCounter: Text-aware Visual Representation for Zero-Shot Object Counting

arXiv: https://arxiv.org/abs/2312.16580 · HTML: https://mlsanity.com/paper/2312.16580/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 27 Dec 2023.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Extract Free Dense Labels from CLIP](https://mlsanity.com/paper/2112.01071/) (2021)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [PPT: Pre-trained Prompt Tuning for Few-shot Learning](https://mlsanity.com/paper/2109.04332/) (2021)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Learning To Count Everything](https://mlsanity.com/paper/2104.08391/) (2021)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [A Closer Look at the Explainability of Contrastive Language-Image Pre-training](https://mlsanity.com/paper/2304.05653/) (2023)
- [A Large Contextual Dataset for Classification, Detection and Counting of Cars with Deep Learning](https://mlsanity.com/paper/1609.04453/) (2016)
- [Class-Agnostic Counting](https://mlsanity.com/paper/1811.00472/) (2018)
- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting](https://mlsanity.com/paper/2203.08354/) (2022)
- [Zero-Shot Temporal Action Detection via Vision-Language Prompting](https://mlsanity.com/paper/2207.08184/) (2022)
- [Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives](https://mlsanity.com/paper/2201.11736/) (2022)
- [Object Counting and Instance Segmentation with Image-level Supervision](https://mlsanity.com/paper/1903.02494/) (2019)
- [Counting Everyday Objects in Everyday Scenes](https://mlsanity.com/paper/1604.03505/) (2016)
- [Pushing the Frontiers of Unconstrained Crowd Counting: New Dataset and Benchmark Method](https://mlsanity.com/paper/1910.12384/) (2019)

## Built on by (most cited first, 30 of 45)

- [CountGD: Multi-Modal Open-World Counting](https://mlsanity.com/paper/2407.04619/) (2024)
- [Zero-shot Object Counting with Good Exemplars](https://mlsanity.com/paper/2407.04948/) (2024)
- [LVLM-COUNT: Enhancing the Counting Ability of Large Vision-Language Models](https://mlsanity.com/paper/2412.00686/) (2024)
- [T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting](https://mlsanity.com/paper/2502.20625/) (2025)
- [CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward](https://mlsanity.com/paper/2504.03724/) (2025)
- [Mind the Prompt: A Novel Benchmark for Prompt-based Class-Agnostic Counting](https://mlsanity.com/paper/2409.15953/) (2024)
- [Open-World Object Counting in Videos](https://mlsanity.com/paper/2506.15368/) (2025)
- [OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videos](https://mlsanity.com/paper/2407.17085/) (2024)
- [Exploring Contextual Attribute Density in Referring Expression Counting](https://mlsanity.com/paper/2503.12460/) (2025)
- [YOLO-Count: Differentiable Object Counting for Text-to-Image Generation](https://mlsanity.com/paper/2508.00728/) (2025)
- [CountGD++: Generalized Prompting for Open-World Counting](https://mlsanity.com/paper/2512.23351/) (2025)
- [A Survey on Class-Agnostic Counting: Advancements from Reference-Based to Open-World Text-Guided Approaches](https://mlsanity.com/paper/2501.19184/) (2025)
- [Can Current AI Models Count What We Mean, Not What They See? A Benchmark and Systematic Evaluation](https://mlsanity.com/paper/2509.13939/) (2025)
- [Single Domain Generalization for Few-Shot Counting via Universal Representation Matching](https://mlsanity.com/paper/2505.16778/) (2025)
- [TFCounter:Polishing Gems for Training-Free Object Counting](https://mlsanity.com/paper/2405.02301/) (2024)
- [SDVPT: Semantic-Driven Visual Prompt Tuning for Open-World Object Counting](https://mlsanity.com/paper/2504.17395/) (2025)
- [What is the Right Embedding Space for Contrastive Learning in Referring Expression Counting?](https://mlsanity.com/paper/2505.22850/) (2025)
- [Text-promptable Object Counting via Quantity Awareness Enhancement](https://mlsanity.com/paper/2507.06679/) (2025)
- [Chain-of-Look Spatial Reasoning for Dense Surgical Instrument Counting](https://mlsanity.com/paper/2602.11024/) (2026)
- [Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting](https://mlsanity.com/paper/2602.12774/) (2026)
- [Does it Really Count? Assessing Semantic Grounding in Text-Guided Class-Agnostic Counting](https://mlsanity.com/paper/2605.02752/) (2026)
- [Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark](https://mlsanity.com/paper/2411.13056/) (2024)
- [Expanding Zero-Shot Object Counting with Rich Prompts](https://mlsanity.com/paper/2505.15398/) (2025)
- [Querying Autonomous Vehicle Point Clouds: Enhanced by 3D Object Counting with CounterNet](https://mlsanity.com/paper/2507.19209/) (2025)
- [Decoupling What to Count and Where to See for Referring Expression Counting](https://mlsanity.com/paper/2510.24374/) (2025)
- [Counting Through Occlusion: Framework for Open World Amodal Counting](https://mlsanity.com/paper/2511.12702/) (2025)
- [Object Counting with GPT-4o and GPT-5: A Comparative Study](https://mlsanity.com/paper/2512.03233/) (2025)
- [CountEx: Fine-Grained Counting via Exemplars and Exclusion](https://mlsanity.com/paper/2602.19432/) (2026)
- [Count Anything at Any Granularity](https://mlsanity.com/paper/2605.10887/) (2026)
- [AdaCount: Training-Free Similarity-Guided Spatial and Feature Adaptation for Zero-Shot Object Counting](https://mlsanity.com/paper/2607.02139/) (2026)

## Abstract (arXiv)

Zero-Shot Object Counting (ZSOC) aims to count referred instances of arbitrary classes in a query image without human-annotated exemplars. To deal with ZSOC, preceding studies proposed a two-stage pipeline: discovering exemplars and counting. However, there remains a challenge of vulnerability to error propagation of the sequentially designed two-stage process. In this work, an one-stage baseline, Visual-Language Baseline (VLBase), exploring the implicit association of the semantic-patch embeddings of CLIP is proposed. Subsequently, the extension of VLBase to Visual-language Counter (VLCounter) is achieved by incorporating three modules devised to tailor VLBase for object counting. First, Semantic-conditioned Prompt Tuning (SPT) is introduced within the image encoder to acquire target-highlighted representations. Second, Learnable Affine Transformation (LAT) is employed to translate the semantic-patch similarity map to be appropriate for the counting task. Lastly, the layer-wisely encoded features are transferred to the decoder through Segment-aware Skip Connection (SaSC) to keep the generalization capability for unseen classes. Through extensive experiments on FSC147, CARPK, and PUCPR+, the benefits of the end-to-end framework, VLCounter, are demonstrated.