mlsanityData to 7 Oct 2026

MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems

arXiv 2410.14179 ↗First posted 18 Oct 2024cs.CL cs.CVcited by 24

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated impressive abilities across various tasks, including visual question answering and chart comprehension, yet existing benchmarks for chart-related tasks fall short in capturing the complexity of real-world multi-chart scenarios. Current benchmarks primarily focus on single-chart tasks, neglecting the multi-hop reasoning required to extract and integrate information from multiple charts, which is essential in practical applications. To fill this gap, we introduce MultiChartQA, a benchmark that evaluates MLLMs' capabilities in four key areas: direct question answering, parallel question answering, comparative reasoning, and sequential reasoning. Our evaluation of a wide range of MLLMs reveals significant performance gaps compared to humans. These results highlight the challenges in multi-chart comprehension and the potential of MultiChartQA to drive advancements in this field. Our code and data are available at https://github.com/Zivenzhu/Multi-chart-QA

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 51 references that are in the map. First posted on arXiv 18 Oct 2024.

Builds on

30 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Charts26 Dec
  2. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning15 Nov
  3. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering19 Dec

Other references · 27

  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  7. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  8. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  10. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  11. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  12. 2018nocaps: novel object captioning at scalecited by 444
15 more
  1. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  2. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  3. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  4. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  5. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  6. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 137
  7. 2024ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wildcited by 79
  8. 2024ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoningcited by 68
  9. 2023PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuningcited by 39
  10. 2024ReMI: A Dataset for Reasoning with Multiple Imagescited by 27
  11. 2023Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Modelscited by 21
  12. 2024ChartThinker: A Contextual Chain-of-Thought Approach to Optimized Chart Summarizationcited by 19
  13. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Taskscited by 18
  14. 2024Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Trainingcited by 17
  15. 2024Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoningcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 12; 2026: 11.

24 papers.

Sort:

By area:

  1. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  2. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  3. 2025DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understandingcited by 11
  4. 2025MuSciClaims: Multimodal Scientific Claim Verificationcited by 8
  5. 2025VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMscited by 7
  6. 2025ChartGen: Scaling Chart Understanding Via Code-Guided Synthetic Chart Generationcited by 7
  7. 2024MageBench: Bridging Large Multimodal Models to Agentscited by 6
  8. 2025Chart Question Answering from Real-World Analytical Narrativescited by 3
  9. 2025DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboardscited by 3
  10. 2025ChartAB: A Benchmark for Chart Grounding & Dense Alignmentcited by 3
  11. 2026Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewardscited by 3
  12. 2026ChartDiff: A Large-Scale Benchmark for Comprehending Pairs of Chartscited by 2
12 more
  1. 2025GuessBench: Sensemaking Multimodal Creativity in the Wildcited by 1
  2. 2025ChartAnchor: Chart Grounding with Structural-Semantic Fidelitycited by 1
  3. 2026Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancementcited by 1
  4. 2026Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledgecited by 1
  5. 2026AnnoBench: A Benchmark for Visualization Annotation Generationcited by 1
  6. 2025Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Taskscited by 0
  7. 2026Beyond Single Plots: A Benchmark for Question Answering on Multi-Chartscited by 0
  8. 2026SD-GRPO: Verifiable Segment Decomposition for Long-Form Vision-Language Generationcited by 0
  9. 2026Benchmarking Multimodal Large Language Models for Scientific Visualization Literacycited by 0
  10. 2026ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generationcited by 0
  11. 2026How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figurescited by 0
  12. 2026LegendBench: A Diagnostic Benchmark for Legend Understanding with Counterfactual Interventionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering
  2. 2025POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering
  3. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering
  4. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
  5. 2026LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning
  6. 2024mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoning
  7. 2024Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuning
  8. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering