mlsanityData to 7 Oct 2026

R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?

arXiv 2410.05474 ↗First posted 7 Oct 2024cs.CV cs.MM eess.IVcited by 29

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

The outstanding performance of Large Multimodal Models (LMMs) has made them widely applied in vision-related tasks. However, various corruptions in the real world mean that images will not be as ideal as in simulations, presenting significant challenges for the practical application of LMMs. To address this issue, we introduce R-Bench, a benchmark focused on the **Real-world Robustness of LMMs**. Specifically, we: (a) model the complete link from user capture to LMMs reception, comprising 33 corruption dimensions, including 7 steps according to the corruption sequence, and 7 groups based on low-level attributes; (b) collect reference/distorted image dataset before/after corruption, including 2,970 question-answer pairs with human labeling; (c) propose comprehensive evaluation for absolute/relative robustness and benchmark 20 mainstream LMMs. Results show that while LMMs can correctly handle the original reference images, their performance is not stable when faced with distorted images, and there is a significant gap in robustness compared to the human visual system. We hope that R-Bench will inspire improving the robustness of LMMs, **extending them from experimental simulations to the real-world application**. Check https://q-future.github.io/R-Bench for details.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 59 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

45 of 59 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  2. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phone3 Aug
  4. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models10 Jul
  5. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model29 Jan
  6. 2024Benchmarking Large Multimodal Models against Common Corruptions22 Jan
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks21 Dec
  8. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captions21 Nov
  9. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models11 Nov
  10. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration7 Nov
  11. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  12. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
2 more
  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning11 May

Other references · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  10. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  11. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  12. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
19 more
  1. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  2. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
  3. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  4. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  5. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
  6. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 255
  7. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  8. 2023TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessmentcited by 220
  9. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  10. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  11. 2024InstructIR: High-Quality Image Restoration Following Human Instructionscited by 139
  12. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 89
  13. 2024Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Modelscited by 64
  14. 2024A-Bench: Are LMMs Masters at Evaluating AI-generated Images?cited by 39
  15. 2022Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shiftcited by 24
  16. 2022Robustness Analysis of Video-Language Models Against Visual and Language Perturbationscited by 23
  17. 2024Q-Refine: A Perceptual Quality Refiner for AI-Generated Imagecited by 21
  18. 2024G-Refine: A General Quality Refiner for Text-to-Image Generationcited by 8
  19. 2024CMC-Bench: Towards a New Paradigm of Visual Signal Compressioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 14; 2026: 13.

29 papers.

Sort:

By area:

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2026Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Modelscited by 21
  5. 2025Redundancy Principles for MLLMs Benchmarkscited by 18
  6. 2025Information Density Principle for MLLM Benchmarkscited by 8
  7. 2025Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Inputcited by 8
  8. 2026What Makes Video World Model Latents Action-Relevant: Prediction over Reconstructioncited by 8
  9. 2025StructVRM: Aligning Multimodal Reasoning with Structured and Verifiable Reward Modelscited by 6
  10. 2025OmniBridge: Unified Multimodal Understanding, Generation, and Retrieval via Latent Space Alignmentcited by 5
  11. 2025Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Modelscited by 5
  12. 2025Using GUI Agent for Electronic Design Automationcited by 5
17 more
  1. 2025AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Modelcited by 4
  2. 2025Diagnosing Corruption-Induced Reliability Failures in Vision-Language Modelscited by 3
  3. 2024Embodied Image Quality Assessment for Robotic Intelligencecited by 2
  4. 2026RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenescited by 2
  5. 2025Robust Driving QA through Metadata-Grounded Context and Task-Specific Promptscited by 1
  6. 2025jina-vlm: Small Multilingual Vision Language Modelcited by 1
  7. 2026CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Modelscited by 1
  8. 2026Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Modelscited by 1
  9. 2026Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?cited by 1
  10. 2026Confidence-Aware Tool Orchestration for Robust Video Understandingcited by 1
  11. 2025DEVAL: A Framework for Evaluating and Improving the Derivation Capability of Large Language Modelscited by 0
  12. 2026MedQ-Deg: A Multidimensional Benchmark for Evaluating MLLMs Across Medical Image Quality Degradationscited by 0
  13. 2026Robust Test-time Video-Text Retrieval: Benchmarking and Adapting for Query Shiftscited by 0
  14. 2026Latent Denoising Improves Visual Alignment in Large Multimodal Modelscited by 0
  15. 2026Learning from Fine-Grained Visual Discrepancies: Mitigating Multimodal Hallucinations via In-Context Visual Contrastive Optimizationcited by 0
  16. 2026BRUCE: Benchmarking Robustness Under Corruption Escalation for Scientific Vision-Language Reasoningcited by 0
  17. 2026Reading Right, Answering Wrong: How Visual Configuration Changes Affect Evidence Use in VLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models
  2. 2025ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
  3. 2025Benchmarking the Trustworthiness in Multimodal LLMs for Video Understanding
  4. 2024GRAB: A Challenging GRaph Analysis Benchmark for Large Multimodal Models
  5. 2026LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models
  6. 2023Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models
  7. 2026VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models
  8. 2025VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation