mlsanityData to 7 Oct 2026

UNICBench: UNIfied Counting Benchmark for MLLM

arXiv 2603.00595 ↗First posted 28 Feb 2026cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Counting is a core capability for multimodal large language models (MLLMs), yet there is no unified counting dataset to rigorously evaluate this ability across image, text, and audio. We present UNICBench, a unified multimodal, multi level counting benchmark and evaluation toolkit with accurate ground truth, deterministic numeric parsing, and stratified reporting. The corpus comprises 5,300 images (5,508 QA), 872 documents (5,888 QA), and 2,069 audio clips (2,905 QA), annotated with a three level capability taxonomy and difficulty tags. Under a standardized protocol with fixed splits/prompts/seeds and modality specific matching rules, we evaluate 45 state-of-the-art MLLMs across modalities. Results show strong performance on some basic counting tasks but significant gaps on reasoning and the hardest partitions, highlighting long-tail errors and substantial headroom for improving general counting. UNICBench offers a rigorous and comparable basis for measurement and a public toolkit to accelerate progress.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 65 references that are in the map. First posted on arXiv 28 Feb 2026.

Builds on

45 of 65 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025VQArt-Bench: A semantically rich VQA Benchmark for Art and Cultural Heritage14 Oct
  2. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb

Other references · 43

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  9. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  10. 2025Qwen2.5-Omni Technical Reportcited by 704
  11. 2024Qwen2-Audio Technical Reportcited by 543
  12. 2025Qwen3-Omni Technical Reportcited by 451
31 more
  1. 2018nocaps: novel object captioning at scalecited by 444
  2. 2018CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenescited by 267
  3. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  4. 2018Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowdscited by 186
  5. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 179
  6. 2017Drone-based Object Counting by Spatially Regularized Regional Proposal Networkcited by 155
  7. 2021Rethinking Counting and Localization in Crowds:A Purely Point-Based Frameworkcited by 152
  8. 2020NWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localizationcited by 151
  9. 2021Learning To Count Everythingcited by 149
  10. 2019Learning from Synthetic Data for Crowd Counting in the Wildcited by 148
  11. 2020JHU-CROWD++: Large-Scale Crowd Counting Dataset and A Benchmark Methodcited by 109
  12. 2022An End-to-End Transformer Model for Crowd Localizationcited by 74
  13. 2019Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detectioncited by 73
  14. 2020Localization in the Crowd with Topological Constraintscited by 68
  15. 2021M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challengecited by 67
  16. 2020Global Wheat Head Detection (GWHD) dataset: a large and diverse dataset of high resolution RGB labelled images to develop and benchmark wheat head detection methodscited by 42
  17. 2023STEERER: Resolving Scale Variations for Counting and Localization via Selective Inheritance Learningcited by 41
  18. 2018Learning to Count Objects in Natural Images for Visual Question Answeringcited by 40
  19. 2019PCC Net: Perspective Crowd Counting via Spatial Convolutional Networkcited by 33
  20. 2025MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Researchcited by 30
  21. 2024Improving Point-based Crowd Counting and Localization Based on Auxiliary Point Guidancecited by 22
  22. 2024LOVA3: Learning to Visual Question Answering, Asking and Assessmentcited by 21
  23. 2022DR.VIC: Decomposition and Reasoning for Video Individual Countingcited by 18
  24. 2024NWPU-MOC: A Benchmark for Fine-grained Multi-category Object Counting in Aerial Imagescited by 14
  25. 2025Vision-Language Models Struggle to Align Entities across Modalitiescited by 13
  26. 2023Zero-Shot Object Counting with Language-Vision Modelscited by 9
  27. 2025WikiMixQA: A Multimodal Benchmark for Question Answering over Tables and Chartscited by 8
  28. 2025Video Individual Counting for Moving Dronescited by 4
  29. 2023Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approachcited by 2
  30. 2023TACNET: Temporal Audio Source Counting Networkcited by 2
  31. 2024Dynamic Proxy Domain Generalizes the Crowd Localization by Better Binary Segmentationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HoloCount: A Holistic Visual Counting Benchmark for MLLMscited by 0
  2. 2026Object Counting Across Modalities: Taxonomies, Benchmarks, Applications, and Open Challengescited by 0
  3. 2026VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CountQA: How Well Do MLLMs Count in the Wild?
  2. 2026Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack
  3. 2026Counting to Four is still a Chore for VLMs
  4. 2024MileBench: Benchmarking MLLMs in Long Context
  5. 2025Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting
  6. 2026Diagnosing Long-Video Quantitative Reasoning in Multimodal LLMs via Enumeration and Counting
  7. 2024LVLM-COUNT: Enhancing the Counting Ability of Large Vision-Language Models
  8. 2026The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs