# Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting

arXiv: https://arxiv.org/abs/2305.04440 · HTML: https://mlsanity.com/paper/2305.04440/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 8 May 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [Learning To Count Everything](https://mlsanity.com/paper/2104.08391/) (2021)
- [Class-Agnostic Counting](https://mlsanity.com/paper/1811.00472/) (2018)
- [Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting](https://mlsanity.com/paper/2203.08354/) (2022)
- [CounTR: Transformer-based Generalised Visual Counting](https://mlsanity.com/paper/2208.13721/) (2022)
- [An End-to-End Transformer Model for Crowd Localization](https://mlsanity.com/paper/2202.13065/) (2022)
- [Boosting Crowd Counting via Multifaceted Attention](https://mlsanity.com/paper/2203.02636/) (2022)
- [Localization in the Crowd with Topological Constraints](https://mlsanity.com/paper/2012.12482/) (2020)
- [Few-Shot Object Detection with Fully Cross-Transformer](https://mlsanity.com/paper/2203.15021/) (2022)
- [Few-shot Object Counting with Similarity-Aware Feature Enhancement](https://mlsanity.com/paper/2201.08959/) (2022)
- [TransCrowd: weakly-supervised crowd counting with transformers](https://mlsanity.com/paper/2104.09116/) (2021)
- [Where are the Blobs: Counting by Localization with Point Supervision](https://mlsanity.com/paper/1807.09856/) (2018)
- [Rethinking Spatial Invariance of Convolutional Networks for Object Counting](https://mlsanity.com/paper/2206.05253/) (2022)
- [MixFormer: Mixing Features across Windows and Dimensions](https://mlsanity.com/paper/2204.02557/) (2022)
- [TasselNet: Counting maize tassels in the wild via local counts regression network](https://mlsanity.com/paper/1707.02290/) (2017)
- [Efficient Crowd Counting via Structured Knowledge Transfer](https://mlsanity.com/paper/2003.10120/) (2020)
- [Coarse to Fine: Domain Adaptive Crowd Counting via Adversarial Scoring Network](https://mlsanity.com/paper/2107.12858/) (2021)

## Built on by (most cited first, 25 of 25)

- [Zero-shot Object Counting with Good Exemplars](https://mlsanity.com/paper/2407.04948/) (2024)
- [Mind the Prompt: A Novel Benchmark for Prompt-based Class-Agnostic Counting](https://mlsanity.com/paper/2409.15953/) (2024)
- [Exploring Contextual Attribute Density in Referring Expression Counting](https://mlsanity.com/paper/2503.12460/) (2025)
- [A Survey on Class-Agnostic Counting: Advancements from Reference-Based to Open-World Text-Guided Approaches](https://mlsanity.com/paper/2501.19184/) (2025)
- [Can Current AI Models Count What We Mean, Not What They See? A Benchmark and Systematic Evaluation](https://mlsanity.com/paper/2509.13939/) (2025)
- [CountingDINO: A Training-free Pipeline for Class-Agnostic Counting using Unsupervised Backbones](https://mlsanity.com/paper/2504.16570/) (2025)
- [SQLNet: Scale-Modulated Query and Localization Network for Few-Shot Class-Agnostic Counting](https://mlsanity.com/paper/2311.10011/) (2023)
- [Generalized-Scale Object Counting with Gradual Query Aggregation](https://mlsanity.com/paper/2511.08048/) (2025)
- [SDVPT: Semantic-Driven Visual Prompt Tuning for Open-World Object Counting](https://mlsanity.com/paper/2504.17395/) (2025)
- [Video Individual Counting With Implicit One-to-Many Matching](https://mlsanity.com/paper/2506.13067/) (2025)
- [TasselNetV4: A vision foundation model for cross-scene, cross-scale, and cross-species plant counting](https://mlsanity.com/paper/2509.20857/) (2025)
- [Crowded Video Individual Counting Informed by Social Grouping and Spatial-Temporal Displacement Priors](https://mlsanity.com/paper/2601.01192/) (2026)
- [Plant Taxonomy Meets Plant Counting: A Fine-Grained, Taxonomic Dataset for Counting Hundreds of Plant Species](https://mlsanity.com/paper/2603.21229/) (2026)
- [Does it Really Count? Assessing Semantic Grounding in Text-Guided Class-Agnostic Counting](https://mlsanity.com/paper/2605.02752/) (2026)
- [Expanding Zero-Shot Object Counting with Rich Prompts](https://mlsanity.com/paper/2505.15398/) (2025)
- [Decoupling What to Count and Where to See for Referring Expression Counting](https://mlsanity.com/paper/2510.24374/) (2025)
- [Object Counting with GPT-4o and GPT-5: A Comparative Study](https://mlsanity.com/paper/2512.03233/) (2025)
- [CoDi -- an exemplar-conditioned diffusion model for low-shot counting](https://mlsanity.com/paper/2512.20153/) (2025)
- [CountEx: Fine-Grained Counting via Exemplars and Exclusion](https://mlsanity.com/paper/2602.19432/) (2026)
- [GCA-SUNet: A Gated Context-Aware Swin-UNet for Exemplar-Free Counting](https://mlsanity.com/paper/2409.12249/) (2024)
- [FocalCount: Towards Class-Count Imbalance in Class-Agnostic Counting](https://mlsanity.com/paper/2502.10677/) (2025)
- [Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges](https://mlsanity.com/paper/2608.23845/) (2026)
- [LookThere! Sparse Vision by Reinforced Selection](https://mlsanity.com/paper/2609.04698/) (2026)
- [Counting Beyond Instances: A Benchmark for Group-Individual Object Counting](https://mlsanity.com/paper/2609.04716/) (2026)
- [AgriCountDINO: Parameter-Efficient Exemplar-Guided Counting and Localization in Agriculture](https://mlsanity.com/paper/2609.29460/) (2026)

## Abstract (arXiv)

Class-agnostic counting (CAC) aims to count objects of interest from a query image given few exemplars. This task is typically addressed by extracting the features of query image and exemplars respectively and then matching their feature similarity, leading to an extract-then-match paradigm. In this work, we show that CAC can be simplified in an extract-and-match manner, particularly using a vision transformer (ViT) where feature extraction and similarity matching are executed simultaneously within the self-attention. We reveal the rationale of such simplification from a decoupled view of the self-attention. The resulting model, termed CACViT, simplifies the CAC pipeline into a single pretrained plain ViT. Further, to compensate the loss of the scale and the order-of-magnitude information due to resizing and normalization in plain ViT, we present two effective strategies for scale and magnitude embedding. Extensive experiments on the FSC147 and the CARPK datasets show that CACViT significantly outperforms state-of-the art CAC approaches in both effectiveness (23.60% error reduction) and generalization, which suggests CACViT provides a concise and strong baseline for CAC. Code will be available.