# MVP-Bench: Can Large Vision--Language Models Conduct Multi-level Visual Perception Like Humans?

arXiv: https://arxiv.org/abs/2410.04345 · HTML: https://mlsanity.com/paper/2410.04345/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 6 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic](https://mlsanity.com/paper/2306.15195/) (2023)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images](https://mlsanity.com/paper/2303.07274/) (2023)
- [StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data](https://mlsanity.com/paper/2308.10253/) (2023)
- [PCA-Bench: Evaluating Multimodal Large Language Models in Perception-Cognition-Action Chain](https://mlsanity.com/paper/2402.15527/) (2024)
- [Can Large Multimodal Models Uncover Deep Semantics Behind Images?](https://mlsanity.com/paper/2402.11281/) (2024)
- [Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation](https://mlsanity.com/paper/2012.04726/) (2020)
- [MAGID: An Automated Pipeline for Generating Synthetic Multi-modal Datasets](https://mlsanity.com/paper/2403.03194/) (2024)
- [Browse and Concentrate: Comprehending Multimodal Content via prior-LLM Context Fusion](https://mlsanity.com/paper/2402.12195/) (2024)
- [TraVLR: Now You See It, Now You Don't! A Bimodal Dataset for Evaluating Visio-Linguistic Reasoning](https://mlsanity.com/paper/2111.10756/) (2021)

## Built on by (most cited first, 9 of 9)

- [HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation](https://mlsanity.com/paper/2505.11454/) (2025)
- [Do You See Me : A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMs](https://mlsanity.com/paper/2506.02022/) (2025)
- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [A Computational Approach to Visual Metonymy](https://mlsanity.com/paper/2601.17706/) (2026)
- [VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging](https://mlsanity.com/paper/2511.18121/) (2025)
- [Assessing Color Vision Test in Large Vision-language Models](https://mlsanity.com/paper/2507.11153/) (2025)
- [From Drone Imagery to Livability Mapping: AI-powered Environment Perception in Rural China](https://mlsanity.com/paper/2508.21738/) (2025)
- [Visual Room 2.0: Seeing is Not Understanding for MLLMs](https://mlsanity.com/paper/2511.12928/) (2025)
- [ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling](https://mlsanity.com/paper/2610.15166/) (2026)

## Abstract (arXiv)

Humans perform visual perception at multiple levels, including low-level object recognition and high-level semantic interpretation such as behavior understanding. Subtle differences in low-level details can lead to substantial changes in high-level perception. For example, substituting the shopping bag held by a person with a gun suggests violent behavior, implying criminal or violent activity. Despite significant advancements in various multimodal tasks, Large Visual-Language Models (LVLMs) remain unexplored in their capabilities to conduct such multi-level visual perceptions. To investigate the perception gap between LVLMs and humans, we introduce MVP-Bench, the first visual-language benchmark systematically evaluating both low- and high-level visual perception of LVLMs. We construct MVP-Bench across natural and synthetic images to investigate how manipulated content influences model perception. Using MVP-Bench, we diagnose the visual perception of 10 open-source and 2 closed-source LVLMs, showing that high-level perception tasks significantly challenge existing LVLMs. The state-of-the-art GPT-4o only achieves an accuracy of 56% on Yes/No questions, compared with 74% in low-level scenarios. Furthermore, the performance gap between natural and manipulated images indicates that current LVLMs do not generalize in understanding the visual semantics of synthetic images as humans do. Our data and code are publicly available at https://github.com/GuanzhenLi/MVP-Bench.