# Point, Segment and Count: A Generalized Framework for Object Counting

arXiv: https://arxiv.org/abs/2311.12386 · HTML: https://mlsanity.com/paper/2311.12386/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 48 references that are in the map.
- First posted on arXiv 21 Nov 2023.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Open-Vocabulary Object Detection Using Captions](https://mlsanity.com/paper/2011.10678/) (2020)
- [Frustratingly Simple Few-Shot Object Detection](https://mlsanity.com/paper/2003.06957/) (2020)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Open-Vocabulary DETR with Conditional Matching](https://mlsanity.com/paper/2203.11876/) (2022)
- [Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector](https://mlsanity.com/paper/1908.01998/) (2019)
- [A Simple Framework for Open-Vocabulary Segmentation and Detection](https://mlsanity.com/paper/2303.08131/) (2023)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [Learning To Count Everything](https://mlsanity.com/paper/2104.08391/) (2021)
- [A Large Contextual Dataset for Classification, Detection and Counting of Cars with Deep Learning](https://mlsanity.com/paper/1609.04453/) (2016)
- [Class-Agnostic Counting](https://mlsanity.com/paper/1811.00472/) (2018)
- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting](https://mlsanity.com/paper/2203.08354/) (2022)
- [CounTR: Transformer-based Generalised Visual Counting](https://mlsanity.com/paper/2208.13721/) (2022)
- [A Low-Shot Object Counting Network With Iterative Prototype Adaptation](https://mlsanity.com/paper/2211.08217/) (2022)
- [Precise Detection in Densely Packed Scenes](https://mlsanity.com/paper/1904.00853/) (2019)

## Built on by (most cited first, 30 of 39)

- [Zero-shot Object Counting with Good Exemplars](https://mlsanity.com/paper/2407.04948/) (2024)
- [T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting](https://mlsanity.com/paper/2502.20625/) (2025)
- [OmniCount: Multi-label Object Counting with Semantic-Geometric Priors](https://mlsanity.com/paper/2403.05435/) (2024)
- [Open-World Object Counting in Videos](https://mlsanity.com/paper/2506.15368/) (2025)
- [CoHD: A Counting-Aware Hierarchical Decoding Framework for Generalized Referring Expression Segmentation](https://mlsanity.com/paper/2405.15658/) (2024)
- [YOLO-Count: Differentiable Object Counting for Text-to-Image Generation](https://mlsanity.com/paper/2508.00728/) (2025)
- [CountGD++: Generalized Prompting for Open-World Counting](https://mlsanity.com/paper/2512.23351/) (2025)
- [A Survey on Class-Agnostic Counting: Advancements from Reference-Based to Open-World Text-Guided Approaches](https://mlsanity.com/paper/2501.19184/) (2025)
- [Can Current AI Models Count What We Mean, Not What They See? A Benchmark and Systematic Evaluation](https://mlsanity.com/paper/2509.13939/) (2025)
- [CountingDINO: A Training-free Pipeline for Class-Agnostic Counting using Unsupervised Backbones](https://mlsanity.com/paper/2504.16570/) (2025)
- [Single Domain Generalization for Few-Shot Counting via Universal Representation Matching](https://mlsanity.com/paper/2505.16778/) (2025)
- [Can Vision-Language Models Count? A Synthetic Benchmark and Analysis of Attention-Based Interventions](https://mlsanity.com/paper/2511.17722/) (2025)
- [Visual Enumeration Remains Challenging for Multimodal Generative AI](https://mlsanity.com/paper/2402.03328/) (2024)
- [Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing](https://mlsanity.com/paper/2406.18540/) (2024)
- [Generalized-Scale Object Counting with Gradual Query Aggregation](https://mlsanity.com/paper/2511.08048/) (2025)
- [A Simple-but-effective Baseline for Training-free Class-Agnostic Counting](https://mlsanity.com/paper/2403.01418/) (2024)
- [Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2603.18523/) (2026)
- [PerSense: Training-Free Personalized Instance Segmentation in Dense Images](https://mlsanity.com/paper/2405.13518/) (2024)
- [FiGO: Fine-Grained Object Counting without Annotations](https://mlsanity.com/paper/2504.11705/) (2025)
- [What is the Right Embedding Space for Contrastive Learning in Referring Expression Counting?](https://mlsanity.com/paper/2505.22850/) (2025)
- [Towards PerSense++: Advancing Training-Free Personalized Instance Segmentation in Dense Images](https://mlsanity.com/paper/2508.14660/) (2025)
- [Does it Really Count? Assessing Semantic Grounding in Text-Guided Class-Agnostic Counting](https://mlsanity.com/paper/2605.02752/) (2026)
- [Expanding Zero-Shot Object Counting with Rich Prompts](https://mlsanity.com/paper/2505.15398/) (2025)
- [Few-Shot Pattern Detection via Template Matching and Regression](https://mlsanity.com/paper/2508.17636/) (2025)
- [Object Counting with GPT-4o and GPT-5: A Comparative Study](https://mlsanity.com/paper/2512.03233/) (2025)
- [Assessing the Visual Enumeration Abilities of Specialized Counting Architectures and Vision-Language Models](https://mlsanity.com/paper/2512.15254/) (2025)
- [CoDi -- an exemplar-conditioned diffusion model for low-shot counting](https://mlsanity.com/paper/2512.20153/) (2025)
- [OCCAM: Class-Agnostic, Training-Free, Prior-Free and Multi-Class Object Counting](https://mlsanity.com/paper/2601.13871/) (2026)
- [CountEx: Fine-Grained Counting via Exemplars and Exclusion](https://mlsanity.com/paper/2602.19432/) (2026)
- [AdaCount: Training-Free Similarity-Guided Spatial and Feature Adaptation for Zero-Shot Object Counting](https://mlsanity.com/paper/2607.02139/) (2026)

## Abstract (arXiv)

Class-agnostic object counting aims to count all objects in an image with respect to example boxes or class names, a.k.a few-shot and zero-shot counting. In this paper, we propose a generalized framework for both few-shot and zero-shot object counting based on detection. Our framework combines the superior advantages of two foundation models without compromising their zero-shot capability: (i) SAM to segment all possible objects as mask proposals, and (ii) CLIP to classify proposals to obtain accurate object counts. However, this strategy meets the obstacles of efficiency overhead and the small crowded objects that cannot be localized and distinguished. To address these issues, our framework, termed PseCo, follows three steps: point, segment, and count. Specifically, we first propose a class-agnostic object localization to provide accurate but least point prompts for SAM, which consequently not only reduces computation costs but also avoids missing small objects. Furthermore, we propose a generalized object classification that leverages CLIP image/text embeddings as the classifier, following a hierarchical knowledge distillation to obtain discriminative classifications among hierarchical mask proposals. Extensive experimental results on FSC-147, COCO, and LVIS demonstrate that PseCo achieves state-of-the-art performance in both few-shot/zero-shot object counting/detection. Code: https://github.com/Hzzone/PseCo