mlsanityData to 14 Oct 2026

MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models

arXiv 2506.04688 ↗First posted 5 Jun 2025cs.CL cs.AI cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

This paper introduces MMRefine, a MultiModal Refinement benchmark designed to evaluate the error refinement capabilities of Multimodal Large Language Models (MLLMs). As the emphasis shifts toward enhancing reasoning during inference, MMRefine provides a framework that evaluates MLLMs' abilities to detect and correct errors across six distinct scenarios beyond just comparing final accuracy before and after refinement. Furthermore, the benchmark analyzes the refinement performance by categorizing errors into six error types. Experiments with various open and closed MLLMs reveal bottlenecks and factors impeding refinement performance, highlighting areas for improvement in effective reasoning enhancement. Our code and dataset are publicly available at https://github.com/naver-ai/MMRefine.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 26 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

20 of 26 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  3. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug

Other references · 17

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  6. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  7. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  8. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  9. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  10. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
  11. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 139
  12. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 86
5 more
  1. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Datacited by 52
  2. 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correctioncited by 45
  3. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 41
  4. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  5. 2024VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoningcited by 14

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025VLM-SlideEval: Evaluating VLMs on Structured Comprehension and Perturbation Sensitivity in PPTcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026MMPCBench: Benchmarking Multimodal Large Language Models on Proactive Critique of Flawed Inputs
  2. 2025Retrieval Enhanced Feedback via In-context Neural Error-book
  3. 2024MM-InstructEval: Zero-Shot Evaluation of (Multimodal) Large Language Models on Multimodal Reasoning Tasks
  4. 2025Reasoning Multimodal Large Language Model: Data Contamination and Dynamic Evaluation
  5. 2025Can Large Multimodal Models Actively Recognize Faulty Inputs? A Systematic Evaluation Framework of Their Input Scrutiny Ability
  6. 2026Instinct vs. Reflection: Unifying Token and Verbalized Confidence in Multimodal Large Models
  7. 2025Learning to Inference Adaptively for Multimodal Large Language Models
  8. 2025Self-Improvement in Multimodal Large Language Models: A Survey