mlsanityData to 14 Oct 2026

COHERENCE: Benchmarking Fine-Grained Image-Text Alignment in Interleaved Multimodal Contexts

arXiv 2604.27389 ↗First posted 30 Apr 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

In recent years, Multimodal Large Language Models (MLLMs) have achieved remarkable progress on a wide range of multimodal benchmarks. Despite these advances, most existing benchmarks mainly focus on single-image or multi-image comprehension. In real-world scenarios such as document reading, information is often presented as interleaved multimodel contexts. This requires MLLMs not only to recognize the content of individual images, but also to identify relevant textual and visual evidence, establish fine-grained alignments between them, and reason over these aligned signals in interleaved contexts based on contextual evidence. However, there is still a lack of systematic benchmarks for quantifying the fine-grained understanding ability of MLLMs in interleaved image-text contexts. To fill this gap, we propose COHERENCE, a benchmark designed to evaluate the ability of MLLMs to recover fine-grained image-text correspondences in interleaved multimodal contexts. COHERENCE covers interleaved image-text content from four representative domains and contains 6,161 high-quality questions. Moreover, we perform a six-type error analysis, enabling fine-grained attribution of failures in interleaved image-text understanding to the specific capabilities missing in current MLLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 53 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

42 of 53 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Qwen3-VL Technical Report26 Nov
  2. 2024MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models14 Oct
  3. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models5 Aug
  4. 2024MIBench: Evaluating Multimodal Large Language Models over Multiple Images21 Jul
  5. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models10 Jul
  6. 2024ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation8 Jul
  7. 2024MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens17 Jun
  8. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding13 Jun
  9. 2024II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models9 Jun
  10. 2024MileBench: Benchmarking MLLMs in Long Context29 Apr
  11. 2024BLINK: Multimodal Large Language Models Can See but Not Perceive18 Apr
  12. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training14 Mar
4 more
  1. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences19 Jan
  2. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Models28 Nov
  3. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension30 Jul
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun

Other references · 26

  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  7. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  8. 2018Object Hallucination in Image Captioningcited by 669
  9. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  10. 2024Long Context Transfer from Language to Visioncited by 515
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  12. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
14 more
  1. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  2. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  3. 2024What matters when building vision-language models?cited by 326
  4. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 102
  5. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 80
  6. 2026CL-bench: A Benchmark for Context Learningcited by 47
  7. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Modelscited by 46
  8. 2021MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understandingcited by 45
  9. 2026STEP3-VL-10B Technical Reportcited by 32
  10. 2026Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scalecited by 29
  11. 2024Holistic Evaluation for Interleaved Text-and-Image Generationcited by 28
  12. 2024CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generationcited by 23
  13. 2025DocAgent: A Multi-Agent System for Automated Code Documentation Generationcited by 18
  14. 2025Self-Taught Agentic Long Context Understandingcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment
  2. 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review
  3. 2024OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation
  4. 2024Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models
  5. 2025From Easy to Hard: The MIR Benchmark for Progressive Interleaved Multi-Image Reasoning
  6. 2026The Alignment Illusion in Multimodal Large Language Models
  7. 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
  8. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension