# UNICBench: UNIfied Counting Benchmark for MLLM

arXiv: https://arxiv.org/abs/2603.00595 · HTML: https://mlsanity.com/paper/2603.00595/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 45 of its 65 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 30 of 45)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [nocaps: novel object captioning at scale](https://mlsanity.com/paper/1812.08658/) (2018)
- [CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes](https://mlsanity.com/paper/1802.10062/) (2018)
- [SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models](https://mlsanity.com/paper/2307.10635/) (2023)
- [Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds](https://mlsanity.com/paper/1808.01050/) (2018)
- [What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks](https://mlsanity.com/paper/2305.18365/) (2023)
- [Drone-based Object Counting by Spatially Regularized Regional Proposal Network](https://mlsanity.com/paper/1707.05972/) (2017)
- [Rethinking Counting and Localization in Crowds:A Purely Point-Based Framework](https://mlsanity.com/paper/2107.12746/) (2021)
- [NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization](https://mlsanity.com/paper/2001.03360/) (2020)
- [Learning To Count Everything](https://mlsanity.com/paper/2104.08391/) (2021)
- [Learning from Synthetic Data for Crowd Counting in the Wild](https://mlsanity.com/paper/1903.03303/) (2019)
- [JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Method](https://mlsanity.com/paper/2004.03597/) (2020)
- [An End-to-End Transformer Model for Crowd Localization](https://mlsanity.com/paper/2202.13065/) (2022)
- [Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection](https://mlsanity.com/paper/1906.07538/) (2019)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [Localization in the Crowd with Topological Constraints](https://mlsanity.com/paper/2012.12482/) (2020)
- [Global Wheat Head Detection (GWHD) dataset: a large and diverse dataset of high resolution RGB labelled images to develop and benchmark wheat head detection methods](https://mlsanity.com/paper/2005.02162/) (2020)
- [STEERER: Resolving Scale Variations for Counting and Localization via Selective Inheritance Learning](https://mlsanity.com/paper/2308.10468/) (2023)

## Built on by (most cited first, 3 of 3)

- [HoloCount: A Holistic Visual Counting Benchmark for MLLMs](https://mlsanity.com/paper/2607.06420/) (2026)
- [Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challenges](https://mlsanity.com/paper/2608.23845/) (2026)
- [VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation](https://mlsanity.com/paper/2609.37003/) (2026)

## Abstract (arXiv)

Counting is a core capability for multimodal large language models (MLLMs), yet there is no unified counting dataset to rigorously evaluate this ability across image, text, and audio. We present UNICBench, a unified multimodal, multi level counting benchmark and evaluation toolkit with accurate ground truth, deterministic numeric parsing, and stratified reporting. The corpus comprises 5,300 images (5,508 QA), 872 documents (5,888 QA), and 2,069 audio clips (2,905 QA), annotated with a three level capability taxonomy and difficulty tags. Under a standardized protocol with fixed splits/prompts/seeds and modality specific matching rules, we evaluate 45 state-of-the-art MLLMs across modalities. Results show strong performance on some basic counting tasks but significant gaps on reasoning and the hardest partitions, highlighting long-tail errors and substantial headroom for improving general counting. UNICBench offers a rigorous and comparable basis for measurement and a public toolkit to accelerate progress.