mlsanityData to 7 Oct 2026

Exploring and Evaluating Multimodal Knowledge Reasoning Consistency of Multimodal Large Language Models

arXiv 2503.04801 ↗First posted 3 Mar 2025cs.CL cs.AIcited by 8

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

In recent years, multimodal large language models (MLLMs) have achieved significant breakthroughs, enhancing understanding across text and vision. However, current MLLMs still face challenges in effectively integrating knowledge across these modalities during multimodal knowledge reasoning, leading to inconsistencies in reasoning outcomes. To systematically explore this issue, we propose four evaluation tasks and construct a new dataset. We conduct a series of experiments on this dataset to analyze and compare the extent of consistency degradation in multimodal knowledge reasoning within MLLMs. Based on the experimental results, we identify factors contributing to the observed degradation in consistency. Our research provides new insights into the challenges of multimodal knowledge reasoning and offers valuable guidance for future efforts aimed at improving MLLMs.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 31 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

17 of 31 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning18 Jun
  2. 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought26 May
  3. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences19 Jan

Other references · 14

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  6. 2023A Survey on Multimodal Large Language Modelscited by 898
  7. 2023What Makes for Good Visual Tokenizers for Large Language Models?cited by 42
  8. 2024MC-MKE: A Fine-Grained Multimodal Knowledge Editing Benchmark Emphasizing Modality Consistencycited by 22
  9. 2024Cross-Modal Consistency in Multimodal Large Language Modelscited by 17
  10. 2024Lessons from Studying Two-Hop Latent Reasoningcited by 15
  11. 2024Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictionscited by 11
  12. 2024MMRel: Benchmarking Relation Understanding in Multi-Modal Large Language Modelscited by 10
2 more
  1. 2024MM-R3: On (In-)Consistency of Vision-Language Models (VLMs)cited by 10
  2. 2024Multimodal Hypothetical Summary for Retrieval-based Multi-image Question Answeringcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025MATRIX: Mask Track Alignment for Interaction-aware Video Generationcited by 4
  3. 2025IRONIC: Coherence-Aware Reasoning Chains for Multi-Modal Sarcasm Detectioncited by 3
  4. 2025Branch, or Layer? Zeroth-Order Optimization for Continual Learning of Vision-Language Modelscited by 3
  5. 2025MLLM-CTBench: A Benchmark for Continual Instruction Tuning with Reasoning Process Diagnosiscited by 2
  6. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  7. 2026CORAL: Correspondence Alignment for Improved Virtual Try-Oncited by 1
  8. 2026Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Understanding the Role of LLMs in Multimodal Evaluation Benchmarks
  2. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
  3. 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models
  4. 2025Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models
  5. 2025LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
  6. 2024A Survey on Benchmarks of Multimodal Large Language Models
  7. 2023MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Tasks
  8. 2025Rethinking the Text-Vision Reasoning Imbalance in MLLMs through the Lens of Training Recipes