# What is the Right Embedding Space for Contrastive Learning in Referring Expression Counting?

arXiv: https://arxiv.org/abs/2505.22850 · HTML: https://mlsanity.com/paper/2505.22850/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 79 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 30 of 55)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [TallyQA: Answering Complex Counting Questions](https://mlsanity.com/paper/1810.12440/) (2018)
- [Context-Aware Crowd Counting](https://mlsanity.com/paper/1811.10452/) (2018)
- [Learning To Count Everything](https://mlsanity.com/paper/2104.08391/) (2021)
- [Distribution Matching for Crowd Counting](https://mlsanity.com/paper/2009.13077/) (2020)
- [Cluster Contrast for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2103.11568/) (2021)
- [Class-Agnostic Counting](https://mlsanity.com/paper/1811.00472/) (2018)
- [CounTR: Transformer-based Generalised Visual Counting](https://mlsanity.com/paper/2208.13721/) (2022)
- [Referring Expression Comprehension: A Survey of Methods and Datasets](https://mlsanity.com/paper/2007.09554/) (2020)
- [Localization in the Crowd with Topological Constraints](https://mlsanity.com/paper/2012.12482/) (2020)
- [CountGD: Multi-Modal Open-World Counting](https://mlsanity.com/paper/2407.04619/) (2024)
- [Few-shot Object Counting and Detection](https://mlsanity.com/paper/2207.10988/) (2022)
- [DAVE -- A Detect-and-Verify Paradigm for Low-Shot Counting](https://mlsanity.com/paper/2404.16622/) (2024)

## Built on by (most cited first, 2 of 2)

- [[De|Re]constructing VLMs' Reasoning in Counting](https://mlsanity.com/paper/2510.19555/) (2025)
- [Decoupling What to Count and Where to See for Referring Expression Counting](https://mlsanity.com/paper/2510.24374/) (2025)

## Abstract (arXiv)

Referring Expression Counting (REC) requires distinguishing visually similar objects described by fine-grained text cues. Existing methods tackle this via image-text contrastive learning of visual features which aims to distinguish visual features corresponding to positive and negative referring expressions. However, this strategy relies heavily on accurate image-text alignment and is constrained by the limited number of available negative expressions. We argue that these limitations can be avoided by performing contrastive learning entirely in the visual embedding space. To this end, we introduce C-REX, a simple but effective supervised contrastive learning framework that learns fine-grained visual representations by contrasting visual tokens within the same image. By shifting the negative samples from incorrect Referring Expressions to visual tokens from the image, C-REX vastly increases the number of potential negatives, providing richer and more stable supervision. This design leads to stronger fine-grained visual discrimination and better generalization across complex open-world counting settings. C-REX serves as a plug-in module that can be seamlessly applied to any existing REC model without architectural changes. We apply it in three different REC architectures and achieve state-of-the-art results improving by up to 28% in MAE and 24.5% in RMSE. Moreover, we show that our framework is versatile and general, and can be applied to other counting tasks like class-agnostic counting, improving the performance of existing models.