# Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting

arXiv: https://arxiv.org/abs/2203.08354 · HTML: https://mlsanity.com/paper/2203.08354/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Bayesian Loss for Crowd Count Estimation with Point Supervision](https://mlsanity.com/paper/1908.03684/) (2019)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [Learning To Count Everything](https://mlsanity.com/paper/2104.08391/) (2021)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [Indices Matter: Learning to Index for Deep Image Matting](https://mlsanity.com/paper/1908.00672/) (2019)
- [Class-Agnostic Counting](https://mlsanity.com/paper/1811.00472/) (2018)
- [Localization in the Crowd with Topological Constraints](https://mlsanity.com/paper/2012.12482/) (2020)
- [From Open Set to Closed Set: Counting Objects by Spatial Divide-and-Conquer](https://mlsanity.com/paper/1908.06473/) (2019)
- [Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning](https://mlsanity.com/paper/1904.02616/) (2019)
- [Weighing Counts: Sequential Crowd Counting by Reinforcement Learning](https://mlsanity.com/paper/2007.08260/) (2020)
- [Neural Similarity Learning](https://mlsanity.com/paper/1910.13003/) (2019)

## Built on by (most cited first, 30 of 80)

- [T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy](https://mlsanity.com/paper/2403.14610/) (2024)
- [DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding](https://mlsanity.com/paper/2411.14347/) (2024)
- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [CounTR: Transformer-based Generalised Visual Counting](https://mlsanity.com/paper/2208.13721/) (2022)
- [Detect Anything via Next Point Prediction](https://mlsanity.com/paper/2510.12798/) (2025)
- [A Low-Shot Object Counting Network With Iterative Prototype Adaptation](https://mlsanity.com/paper/2211.08217/) (2022)
- [CountGD: Multi-Modal Open-World Counting](https://mlsanity.com/paper/2407.04619/) (2024)
- [Zero-shot Object Counting](https://mlsanity.com/paper/2303.02001/) (2023)
- [DAVE -- A Detect-and-Verify Paradigm for Low-Shot Counting](https://mlsanity.com/paper/2404.16622/) (2024)
- [VLCounter: Text-aware Visual Representation for Zero-Shot Object Counting](https://mlsanity.com/paper/2312.16580/) (2023)
- [Learning to Count Anything: Reference-less Class-agnostic Counting with Weak Supervision](https://mlsanity.com/paper/2205.10203/) (2022)
- [Open-world Text-specified Object Counting](https://mlsanity.com/paper/2306.01851/) (2023)
- [Point, Segment and Count: A Generalized Framework for Object Counting](https://mlsanity.com/paper/2311.12386/) (2023)
- [Training-free Object Counting with Prompts](https://mlsanity.com/paper/2307.00038/) (2023)
- [Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting](https://mlsanity.com/paper/2305.04440/) (2023)
- [Counting Guidance for High Fidelity Text-to-Image Synthesis](https://mlsanity.com/paper/2306.17567/) (2023)
- [Zero-shot Object Counting with Good Exemplars](https://mlsanity.com/paper/2407.04948/) (2024)
- [A Novel Unified Architecture for Low-Shot Counting by Detection and Segmentation](https://mlsanity.com/paper/2409.18686/) (2024)
- [T-Rex: Counting by Visual Prompting](https://mlsanity.com/paper/2311.13596/) (2023)
- [LVLM-COUNT: Enhancing the Counting Ability of Large Vision-Language Models](https://mlsanity.com/paper/2412.00686/) (2024)
- [T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting](https://mlsanity.com/paper/2502.20625/) (2025)
- [Can SAM Count Anything? An Empirical Study on SAM Counting](https://mlsanity.com/paper/2304.10817/) (2023)
- [OmniCount: Multi-label Object Counting with Semantic-Geometric Priors](https://mlsanity.com/paper/2403.05435/) (2024)
- [ABC Easy as 123: A Blind Counter for Exemplar-Free Multi-Class Class-agnostic Counting](https://mlsanity.com/paper/2309.04820/) (2023)
- [AFreeCA: Annotation-Free Counting for All](https://mlsanity.com/paper/2403.04943/) (2024)
- [Open-World Object Counting in Videos](https://mlsanity.com/paper/2506.15368/) (2025)
- [A Fixed-Point Approach to Unified Prompt-Based Counting](https://mlsanity.com/paper/2403.10236/) (2024)
- [OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videos](https://mlsanity.com/paper/2407.17085/) (2024)
- [Exploring Contextual Attribute Density in Referring Expression Counting](https://mlsanity.com/paper/2503.12460/) (2025)
- [Zero-Shot Object Counting with Language-Vision Models](https://mlsanity.com/paper/2309.13097/) (2023)

## Abstract (arXiv)

Class-agnostic counting (CAC) aims to count all instances in a query image given few exemplars. A standard pipeline is to extract visual features from exemplars and match them with query images to infer object counts. Two essential components in this pipeline are feature representation and similarity metric. Existing methods either adopt a pretrained network to represent features or learn a new one, while applying a naive similarity metric with fixed inner product. We find this paradigm leads to noisy similarity matching and hence harms counting performance. In this work, we propose a similarity-aware CAC framework that jointly learns representation and similarity metric. We first instantiate our framework with a naive baseline called Bilinear Matching Network (BMNet), whose key component is a learnable bilinear similarity metric. To further embody the core of our framework, we extend BMNet to BMNet+ that models similarity from three aspects: 1) representing the instances via their self-similarity to enhance feature robustness against intra-class variations; 2) comparing the similarity dynamically to focus on the key patterns of each exemplar; 3) learning from a supervision signal to impose explicit constraints on matching results. Extensive experiments on a recent CAC dataset FSC147 show that our models significantly outperform state-of-the-art CAC approaches. In addition, we also validate the cross-dataset generality of BMNet and BMNet+ on a car counting dataset CARPK. Code is at tiny.one/BMNet