# SITUATE -- Synthetic Object Counting Dataset for VLM training

arXiv: https://arxiv.org/abs/2602.00108 · HTML: https://mlsanity.com/paper/2602.00108/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 17 of 17)

- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [Teaching CLIP to Count to Ten](https://mlsanity.com/paper/2302.12066/) (2023)
- [TallyQA: Answering Complex Counting Questions](https://mlsanity.com/paper/1810.12440/) (2018)
- [CLIP-Count: Towards Text-Guided Zero-Shot Object Counting](https://mlsanity.com/paper/2305.07304/) (2023)
- [Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation data](https://mlsanity.com/paper/2401.17600/) (2024)
- [LVLM-COUNT: Enhancing the Counting Ability of Large Vision-Language Models](https://mlsanity.com/paper/2412.00686/) (2024)
- [Synthetic Data is an Elegant GIFT for Continual Vision-Language Models](https://mlsanity.com/paper/2503.04229/) (2025)
- [Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting](https://mlsanity.com/paper/2510.04401/) (2025)
- [CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward](https://mlsanity.com/paper/2504.03724/) (2025)
- [PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs](https://mlsanity.com/paper/2402.08657/) (2024)
- [Zero-Shot Object Counting with Language-Vision Models](https://mlsanity.com/paper/2309.13097/) (2023)
- [RelationVLM: Making Large Vision-Language Models Understand Visual Relations](https://mlsanity.com/paper/2403.12801/) (2024)
- [Synthetic Data Generation with Large Language Models for Personalized Community Question Answering](https://mlsanity.com/paper/2410.22182/) (2024)
- [Benchmarking Vision Language Models on German Factual Data](https://mlsanity.com/paper/2504.11108/) (2025)
- [SDVPT: Semantic-Driven Visual Prompt Tuning for Open-World Object Counting](https://mlsanity.com/paper/2504.17395/) (2025)
- [VLM@school -- Evaluation of AI image understanding on German middle school knowledge](https://mlsanity.com/paper/2506.11604/) (2025)

## Built on by (most cited first, 2 of 2)

- [The MixCount Dataset: Bridging the Data Gap for Open-Vocabulary Object Counting](https://mlsanity.com/paper/2605.18063/) (2026)
- [Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges](https://mlsanity.com/paper/2608.23845/) (2026)

## Abstract (arXiv)

We present SITUATE, a novel dataset designed for training and evaluating Vision Language Models on counting tasks with spatial constraints. The dataset bridges the gap between simple 2D datasets like VLMCountBench and often ambiguous real-life datasets like TallyQA, which lack control over occlusions and spatial composition. Experiments show that our dataset helps to improve generalization for out-of-distribution images, since a finetune of Qwen VL 2.5 7B on SITUATE improves accuracy on the Pixmo count test data, but not vice versa. We cross validate this by comparing the model performance across established other counting benchmarks and against an equally sized fine-tuning set derived from Pixmo count.