mlsanityData to 14 Oct 2026

MIRCaps: A Large-Scale Mixed-Domain Dataset with Image-Level and Region-Level Captions for Fine-Grained Vision-Language Learning

arXiv 2606.21419 ↗First posted 19 Jun 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Despite recent progress in Vision-Language Models (VLMs), mixed-domain image-caption datasets for both general-purpose and CCTV-based video surveillance systems remain limited. To address this gap, we introduce a large-scale multimodal dataset comprising 141,364 images, 981,947 image-level captions, 1,742,264 region-level captions, and 1,391,779 bounding box annotations. Each image is associated with an average of seven image-level captions describing different aspects of the overall scene, as well as seven region-level captions for each annotated bounding box. These complementary caption types are designed to help VLMs learn fine-grained visual attributes, including object categories, estimated sizes, colors, actions, states, and surrounding environmental context. We demonstrate the effectiveness of the dataset on two important downstream tasks: image captioning and object detection. Experimental results show that lightweight VLMs, including SmolVLM-256M-Instruct, BLIP, BLIP2, and Qwen2.5-VL 3B-Instruct, can be effectively fine-tuned using our dataset. Our dataset and code are publicly available at https://zenodo.org/records/20418601.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 31 references that are in the map. First posted on arXiv 19 Jun 2026.

Builds on

14 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024What's in the Image? A Deep-Dive into the Vision of Vision Language Models26 Nov

Other references · 13

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  5. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  6. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  7. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  8. 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 533
  9. 2018nocaps: novel object captioning at scalecited by 444
  10. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  11. 2018CrowdHuman: A Benchmark for Detecting Human in a Crowdcited by 284
  12. 2023Object Attribute Matters in Visual Question Answeringcited by 2
1 more
  1. 2025OD-VIRAT: A Large-Scale Benchmark for Object Detection in Realistic Surveillance Environmentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DataComp-VLM: Improved Open Datasets for Vision-Language Models
  2. 2025Trust the Model: Compact VLMs as In-Context Judges for Image-Text Data Quality
  3. 2024HumanVLM: Foundation for Human-Scene Vision-Language Model
  4. 2023A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions
  5. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions
  6. 2023FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions
  7. 202620/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
  8. 2024Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions