mlsanityData to 14 Oct 2026

Demystifying the Visual Quality Paradox in Multimodal Large Language Models

arXiv 2506.15645 ↗First posted 18 Jun 2025cs.CV cs.AIcited by 3

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Recent Multimodal Large Language Models (MLLMs) excel on benchmark vision-language tasks, yet little is known about how input visual quality shapes their responses. Does higher perceptual quality of images already translate to better MLLM understanding? We conduct the first systematic study spanning leading MLLMs and a suite of vision-language benchmarks, applying controlled degradations and stylistic shifts to each image. Surprisingly, we uncover a visual-quality paradox: model, task, and even individual-instance performance can improve when images deviate from human-perceived fidelity. Off-the-shelf restoration pipelines fail to reconcile these idiosyncratic preferences. To close the gap, we introduce Visual-Quality Test-Time Tuning (VQ-TTT)-a lightweight adaptation module that: (1) inserts a learnable, low-rank kernel before the frozen vision encoder to modulate frequency content; and (2) fine-tunes only shallow vision-encoder layers via LoRA. VQ-TTT dynamically adjusts each input image in a single forward pass, aligning it with task-specific model preferences. Across the evaluated MLLMs and all datasets, VQ-TTT lifts significant average accuracy, with no external models, cached features, or extra training data. These findings redefine ``better'' visual inputs for MLLMs and highlight the need for adaptive, rather than universally ``clean'', imagery, in the new era of AI being the main data customer.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 76 references that are in the map. First posted on arXiv 18 Jun 2025.

Builds on

58 of 76 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs3 Mar
  2. 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs24 Feb
  3. 2025Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization18 Feb
  4. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding13 Dec
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution18 Sep
  6. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models10 Jul
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun

Other references · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2025Qwen2.5-VL Technical Reportcited by 5,587
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  11. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  12. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
38 more
  1. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  5. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  6. 2019Towards VQA Models That Can Readcited by 1,718
  7. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  10. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,358
  11. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  12. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  13. 2017Deep Image Priorcited by 1,098
  14. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  15. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  16. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  17. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 727
  18. 2022Simple Baselines for Image Restorationcited by 708
  19. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  20. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  21. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  22. 2018nocaps: novel object captioning at scalecited by 444
  23. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  24. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  25. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  26. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 294
  27. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 286
  28. 2023Image Restoration with Mean-Reverting Stochastic Differential Equationscited by 183
  29. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  30. 2023Controlling Vision-Language Models for Multi-Task Image Restorationcited by 110
  31. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  32. 2023Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioningcited by 41
  33. 2025Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasetscited by 40
  34. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  35. 2025Can Large Vision Language Models Read Maps Like a Human?cited by 30
  36. 2025SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systemscited by 25
  37. 2024MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformerscited by 24
  38. 2025V2X-UniPool: Unifying Multimodal Perception and Knowledge Reasoning for Autonomous Drivingcited by 14

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Diagnosing Corruption-Induced Reliability Failures in Vision-Language Modelscited by 4
  2. 2026One Token, Two Fates: A Unified Framework via Vision Token Manipulation Against MLLMs Hallucinationcited by 2
  3. 2026Hard to Read, Easy to Jailbreak: How Visual Degradation Bypasses MLLM Safety Alignmentcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models
  2. 2026Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift
  3. 2026Temporal Gains, Spatial Costs: Revisiting Video Fine-Tuning in Multimodal Large Language Models
  4. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
  5. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
  6. 2026Scaling Vision-Language Models Is Not Enough to Mitigate Bias
  7. 2025Efficiently Integrate Large Language Models with Visual Perception: A Survey from the Training Paradigm Perspective
  8. 2026OddGridBench: Exposing the Lack of Fine-Grained Visual Discrepancy Sensitivity in Multimodal Large Language Models