mlsanityData to 14 Oct 2026

Visual Room 2.0: Seeing is Not Understanding for MLLMs

arXiv 2511.12928 ↗First posted 17 Nov 2025cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Can multi-modal large language models (MLLMs) truly understand what they can see? Extending Searle's Chinese Room into the multi-modal domain, this paper proposes the Visual Room argument: MLLMs may describe every visual detail precisely yet fail to comprehend the underlying emotions and intentions, namely seeing is not understanding. Building on this, we introduce textitVisual Room 2.0, a hierarchical benchmark for evaluating perception-cognition alignment of MLLMs. We model human perceptive and cognitive processes across three levels: low, middle, and high, covering 17 representative tasks. The perception component ranges from attribute recognition to scene understanding, while the cognition component extends from textual entailment to causal and social reasoning. The dataset contains 350 multi-modal samples, each with six progressive questions (2,100 in total) spanning perception to cognition. Evaluating 10 state-of-the-art (SoTA) MLLMs, we highlight three key findings: (1) MLLMs exhibit stronger perceptual competence than cognitive ability (8.0%uparrow); (2) cognition appears not causally dependent on perception-based reasoning; and (3) cognition scales with model size, but perception does not consistently improve with larger variants. This work operationalizes Seeing ne Understanding as a testable hypothesis, offering a new paradigm from perceptual processing to cognitive reasoning in MLLMs. Our dataset is available at https://huggingface.co/datasets/LHK2003/PCBench.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 4 of its 15 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

4 of 15 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Can Large Multimodal Models Uncover Deep Semantics Behind Images?17 Feb
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 2

  1. 2024PCA-Bench: Evaluating Multimodal Large Language Models in Perception-Cognition-Action Chaincited by 29
  2. 2024MVP-Bench: Can Large Vision--Language Models Conduct Multi-level Visual Perception Like Humans?cited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Multimodal LLM Augmented Reasoning for Interpretable Visual Perception Analysis
  2. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
  3. 2025Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs
  4. 2025Do You See Me : A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMs
  5. 2025The Perceptual Observatory Characterizing Robustness and Grounding in MLLMs
  6. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
  7. 2026PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models
  8. 2025Evaluating Graphical Perception with Multimodal LLMs