mlsanityData to 7 Oct 2026

Evaluating Graphical Perception with Multimodal LLMs

arXiv 2504.04221 ↗First posted 5 Apr 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have remarkably progressed in analyzing and understanding images. Despite these advancements, accurately regressing values in charts remains an underexplored area for MLLMs. For visualization, how do MLLMs perform when applied to graphical perception tasks? Our paper investigates this question by reproducing Cleveland and McGill's seminal 1984 experiment and comparing it against human task performance. Our study primarily evaluates fine-tuned and pretrained models and zero-shot prompting to determine if they closely match human graphical perception. Our findings highlight that MLLMs outperform human task performance in some cases but not in others. We highlight the results of all experiments to foster an understanding of where MLLMs succeed and fail when applied to data visualization.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 29 references that are in the map. First posted on arXiv 5 Apr 2025.

Builds on

21 of 29 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyond8 Oct
  2. 2024M2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning24 Sep
  3. 2024MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct9 Sep
  4. 2024Exploring Perceptual Limitation of Multimodal Large Language Models12 Feb
  5. 2023What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning2 Nov

Other references · 16

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2015VQA: Visual Question Answeringcited by 2,489
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  4. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  5. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  6. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  7. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  8. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 158
  9. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuningcited by 149
  10. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Modelscited by 111
  11. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 95
  12. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuningcited by 79
4 more
  1. 2024Multimodal Reasoning with Multimodal Knowledge Graphcited by 43
  2. 2024Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuningcited by 27
  3. 2024Understanding Graphical Perception in Data Visualization through Zero-shot Prompting of Vision-Language Modelscited by 6
  4. 2023Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Can a Neural Encoding Model Replicate an fMRI Visualization Study?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Understanding Graphical Perception in Large Multimodal Models
  2. 2024Visualization Literacy of Multimodal Large Language Models: A Comparative Study
  3. 2025Chart-to-Experience: Benchmarking Multimodal LLMs for Predicting Experiential Impact of Charts
  4. 2025Exploring MLLMs Perception of Network Visualization Principles
  5. 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment
  6. 2025I Spy With My Model's Eye: Visual Search as a Behavioural Test for MLLMs
  7. 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach
  8. 2025The Plot Thickens: Quantitative Part-by-Part Exploration of MLLM Visualization Literacy