# CLIP-EBC: CLIP Can Count Accurately through Enhanced Blockwise Classification

arXiv: https://arxiv.org/abs/2403.09281 · HTML: https://mlsanity.com/paper/2403.09281/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 41 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method](https://mlsanity.com/paper/2004.03597/) (2020)
- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [An End-to-End Transformer Model for Crowd Localization](https://mlsanity.com/paper/2202.13065/) (2022)
- [From Open Set to Closed Set: Counting Objects by Spatial Divide-and-Conquer](https://mlsanity.com/paper/1908.06473/) (2019)
- [CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model](https://mlsanity.com/paper/2304.04231/) (2023)
- [CNN-based Cascaded Multi-task Learning of High-level Prior and Density Estimation for Crowd Counting](https://mlsanity.com/paper/1707.09605/) (2017)
- [STEERER: Resolving Scale Variations for Counting and Localization via Selective Inheritance Learning](https://mlsanity.com/paper/2308.10468/) (2023)
- [Point-Query Quadtree for Crowd Counting, Localization, and More](https://mlsanity.com/paper/2308.13814/) (2023)
- [Towards Using Count-level Weak Supervision for Crowd Counting](https://mlsanity.com/paper/2003.00164/) (2020)
- [Encoder-Decoder Based Convolutional Neural Networks with Multi-Scale-Aware Modules for Crowd Counting](https://mlsanity.com/paper/2003.05586/) (2020)
- [Uniformity in Heterogeneity:Diving Deep into Count Interval Partition for Crowd Counting](https://mlsanity.com/paper/2107.12619/) (2021)
- [Discrete-Constrained Regression for Local Counting Models](https://mlsanity.com/paper/2207.09865/) (2022)

## Built on by (most cited first, 12 of 12)

- [On the Limitations of Vision-Language Models in Understanding Image Transforms](https://mlsanity.com/paper/2503.09837/) (2025)
- [On the rankability of visual embeddings](https://mlsanity.com/paper/2507.03683/) (2025)
- [ZIP: Scalable Crowd Counting via Zero-Inflated Poisson Modeling](https://mlsanity.com/paper/2506.19955/) (2025)
- [Semi-Supervised Multi-View Crowd Counting by Ranking Multi-View Fusion Models](https://mlsanity.com/paper/2512.16243/) (2025)
- [Car Object Counting and Position Estimation via Extension of the CLIP-EBC Framework](https://mlsanity.com/paper/2507.08240/) (2025)
- [Local Information Matters: A Rethink of Crowd Counting](https://mlsanity.com/paper/2508.16970/) (2025)
- [Rank-Aware Agglomeration of Foundation Models for Immunohistochemistry Image Cell Counting](https://mlsanity.com/paper/2511.12588/) (2025)
- [Learning Where to Focus: Density-Driven Guidance for Detecting Dense Tiny Objects](https://mlsanity.com/paper/2512.22949/) (2025)
- [Dense Point-to-Mask Optimization with Reinforced Point Selection for Crowd Instance Segmentation](https://mlsanity.com/paper/2604.01742/) (2026)
- [Thermal-Only Crowd Counting with Deployment-Time Privacy Protection](https://mlsanity.com/paper/2605.17042/) (2026)
- [DCA-MoE: Spatially Adaptive Cross-Layer Fusion and Density-Routed Experts for Crowd Counting](https://mlsanity.com/paper/2608.15213/) (2026)
- [CrowdCue: Specialist-Cue Conditioning for Vision-Language Crowd Counting](https://mlsanity.com/paper/2609.23012/) (2026)

## Abstract (arXiv)

We propose CLIP-EBC, the first fully CLIP-based model for accurate crowd density estimation. While the CLIP model has demonstrated remarkable success in addressing recognition tasks such as zero-shot image classification, its potential for counting has been largely unexplored due to the inherent challenges in transforming a regression problem, such as counting, into a recognition task. In this work, we investigate and enhance CLIP's ability to count, focusing specifically on the task of estimating crowd sizes from images. Existing classification-based crowd-counting frameworks have significant limitations, including the quantization of count values into bordering real-valued bins and the sole focus on classification errors. These practices result in label ambiguity near the shared borders and inaccurate prediction of count values. Hence, directly applying CLIP within these frameworks may yield suboptimal performance. To address these challenges, we first propose the Enhanced Blockwise Classification (EBC) framework. Unlike previous methods, EBC utilizes integer-valued bins, effectively reducing ambiguity near bin boundaries. Additionally, it incorporates a regression loss based on density maps to improve the prediction of count values. Within our backbone-agnostic EBC framework, we then introduce CLIP-EBC to fully leverage CLIP's recognition capabilities for this task. Extensive experiments demonstrate the effectiveness of EBC and the competitive performance of CLIP-EBC. Specifically, our EBC framework can improve existing classification-based methods by up to 44.5% on the UCF-QNRF dataset, and CLIP-EBC achieves state-of-the-art performance on the NWPU-Crowd test set, with an MAE of 58.2 and an RMSE of 268.5, representing improvements of 8.6% and 13.3% over the previous best method, STEERER. The code and weights are available at https://github.com/Yiming-M/CLIP-EBC.