mlsanityData to 14 Oct 2026

Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Input

arXiv 2510.16926 ↗First posted 19 Oct 2025cs.CV cs.CLcited by 9

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) increasingly support dynamic image resolutions. However, current evaluation paradigms primarily assess semantic performance, overlooking the critical question of resolution robustness - whether performance remains stable across varying input resolutions. To address this gap, we introduce Res-Bench, a comprehensive benchmark comprising 14,400 samples across 12 resolution levels and six core capability dimensions. We designed a novel evaluation framework that goes beyond traditional accuracy metrics to capture performance stability. This framework introduces multiple robustness metrics: Spearman's correlation for assessing resolution-performance trends, and Absolute/Relative Continuous Error (ACE/RCE) for measuring performance volatility. Using these metrics, we conducted a large-scale evaluation of leading MLLMs. Our analysis encompasses: (1) model-centric and task-centric robustness examination, (2) investigation of preprocessing strategies including padding and super-resolution, and (3) exploration of fine-tuning for stability enhancement.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 37 references that are in the map. First posted on arXiv 19 Oct 2025.

Builds on

27 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding13 Dec
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  3. 2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?7 Oct
  4. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  5. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspective26 May
  6. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension30 Jul

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  10. 2016A Diagram Is Worth A Dozen Imagescited by 804
  11. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  12. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
9 more
  1. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  2. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 255
  3. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  4. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  5. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 108
  6. 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 97
  7. 2024CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacycited by 81
  8. 2025Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometricscited by 23
  9. 2025A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenescited by 3
  2. 2026MedQ-Deg: A Multidimensional Benchmark for Evaluating MLLMs Across Medical Image Quality Degradationscited by 0
  3. 2026How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbationscited by 0
  4. 2026AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoringcited by 0
  5. 2026PEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignmentcited by 0
  6. 2026Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structurescited by 0
  7. 2026Reading Right, Answering Wrong: How Visual Configuration Changes Affect Evidence Use in VLMscited by 0
  8. 2026Can't Find Waldo: Evaluating VLMs' Sensitivity to Image Resolution and Detail Levelcited by 0
  9. 2026DREAM: Dynamic Resolution Assignment For Multimodal Multi-agent Debatecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MileBench: Benchmarking MLLMs in Long Context
  2. 2023SEED-Bench-2: Benchmarking Multimodal Large Language Models
  3. 2024II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models
  4. 2025Evaluating Robustness of Vision-Language Models Under Noisy Conditions
  5. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
  6. 2024Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models
  7. 2026Resolution as a First-Class Decision: Task-Conditioned Routing for Efficient Multimodal Large Language Models
  8. 2025PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications