mlsanityData to 14 Oct 2026

DistortBench: Benchmarking Vision Language Models on Image Distortion Identification

arXiv 2604.19966 ↗First posted 21 Apr 2026cs.CV cs.AI cs.LGcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) are increasingly used in settings where sensitivity to low-level image degradations matters, including content moderation, image restoration, and quality monitoring. Yet their ability to recognize distortion type and severity remains poorly understood. We present DistortBench, a diagnostic benchmark for no-reference distortion perception in VLMs. DistortBench contains 13,500 four-choice questions covering 27 distortion types, six perceptual categories, and five severity levels: 25 distortions inherit KADID-10k calibrations, while two added rotation distortions use monotonic angle-based levels. We evaluate 18 VLMs, including 17 open-weight models from five families and one proprietary model. Despite strong performance on high-level vision-language tasks, the best model reaches only 61.9% accuracy, just below the human majority-vote baseline of 65.7% (average individual: 60.2%), indicating that low-level perceptual understanding remains a major weakness of current VLMs. Our analysis further reveals weak and non-monotonic scaling with model size, performance drops in most base--thinking pairs, and distinct severity-response patterns across model families. We hope DistortBench will serve as a useful benchmark for measuring and improving low-level visual perception in VLMs.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 26 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

18 of 26 references.

References · 18

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  10. 2025Gemma 3 Technical Reportcited by 1,824
  11. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  12. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
6 more
  1. 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 529
  2. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  3. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 249
  4. 2016Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessmentcited by 225
  5. 2020Media Forensics and DeepFakes: an overviewcited by 209
  6. 2023Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Modelscited by 105

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026TASTE: A Designer-Annotated Multi-Dimensional Preference Dataset for AI-Generated Graphic Designcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models
  2. 2025Investigate the Low-level Visual Perception in Vision-Language based Image Quality Assessment
  3. 2025PairBench: Are Vision-Language Models Reliable at Comparing What They See?
  4. 2025On the Domain Robustness of Contrastive Vision-Language Models
  5. 2026Scaling Vision-Language Models Is Not Enough to Mitigate Bias
  6. 2026Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
  7. 2026PRISM-VLM: A Multi-Axis Discriminative Benchmark for Compact Vision-Language Models
  8. 2026MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models