mlsanityData to 14 Oct 2026

SynDORBench: Evaluating LVLM Perceptual Robustness Under Physically Constrained Visibility Conditions

arXiv 2609.31823 ↗First posted 25 Sep 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large vision-language models (LVLMs) have demonstrated remarkable performance on multimodal reasoning benchmarks, yet their perceptual reliability under physically constrained imaging conditions remains poorly understood. Existing evaluations predominantly assume ideal visual inputs and therefore fail to characterize how camera distance, illumination, viewpoint, and pixel density fundamentally affect semantic recoverability. We introduce SynDORBench, the first physically grounded benchmark for evaluating LVLM perceptual robustness under DORI-calibrated conditions aligned with human visual capability standards. SynDORBench comprises over 54k question--answer pairs generated through a controllable synthetic pipeline that systematically varies viewing distance, lighting, camera geometry, and action pose according to physically interpretable pixel-density regimes. To support scalable low-visibility supervision, we further propose a discernibility annotation framework that propagates human perceptual labels using mask-conditioned statistical features and ensemble learning. We evaluate 16 open-source LVLMs, a commercial LVLM baseline, and YOLO11x across human-presence classification and action recognition tasks under progressively degraded visibility conditions. Our results reveal that perceptual failure in LVLMs is strongly governed by pixel density and physical imaging constraints rather than model scale alone. Surprisingly, several compact open-source LVLMs outperform larger commercial baselines and substantially exceed YOLO11x robustness under long-range and low-light conditions. SynDORBench establishes a new benchmark paradigm for physically grounded multimodal evaluation, enabling systematic analysis of LVLM reliability under real-world perceptual constraints and direct comparison against human visibility thresholds.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 32 of its 63 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

32 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 31

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  5. 2025Gemma 3 Technical Reportcited by 1,824
  6. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  9. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  10. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  11. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  12. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
19 more
  1. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 425
  2. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  3. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  4. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 265
  5. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 237
  6. 2024CinePile: A Long Video Question Answering Dataset and Benchmarkcited by 114
  7. 2024Small Language Models: Survey, Measurements, and Insightscited by 112
  8. 2018Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsingcited by 106
  9. 2025PerceptionLM: Open-Access Data and Models for Detailed Visual Understandingcited by 92
  10. 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 80
  11. 2023SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modelingcited by 54
  12. 2024LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Modelscited by 51
  13. 2024THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Modelscited by 36
  14. 2023SMPLitex: A Generative Model and Dataset for 3D Human Texture Estimation from Single Imagecited by 30
  15. 2025SurveillanceVQA-589K: A Benchmark for Comprehensive Surveillance Video-Language Understanding with Large Modelscited by 18
  16. 2024SCaRL- A Synthetic Multi-Modal Dataset for Autonomous Drivingcited by 4
  17. 2024Boundless: Generating Photorealistic Synthetic Data for Object Detection in Urban Streetscapescited by 4
  18. 2021Contrast Limited Adaptive Histogram Equalization (CLAHE) Approach for Enhancement of the Microstructures of Friction Stir Welded Jointscited by 2
  19. 2025Measuring Image-Relation Alignment: Reference-Free Evaluation of VLMs and Synthetic Pre-training for Open-Vocabulary Scene Graph Generationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs
  2. 2025Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forward
  3. 2026Validating, Not Sampling: Region-Level Robustness of Vision-Language and Vision-Language-Action Models
  4. 2025On the Domain Robustness of Contrastive Vision-Language Models
  5. 2025PairBench: Are Vision-Language Models Reliable at Comparing What They See?
  6. 2025The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
  7. 2024UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
  8. 2026VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models