# MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems

arXiv: https://arxiv.org/abs/2410.14179 · HTML: https://mlsanity.com/paper/2410.14179/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 30 of 30)

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge](https://mlsanity.com/paper/2206.01718/) (2022)
- [nocaps: novel object captioning at scale](https://mlsanity.com/paper/1812.08658/) (2018)
- [DVQA: Understanding Data Visualizations via Question Answering](https://mlsanity.com/paper/1801.08163/) (2018)
- [PlotQA: Reasoning over Scientific Plots](https://mlsanity.com/paper/1909.00997/) (2019)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding](https://mlsanity.com/paper/2406.09411/) (2024)
- [LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark](https://mlsanity.com/paper/2306.06687/) (2023)
- [MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning](https://mlsanity.com/paper/2311.10774/) (2023)
- [Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders](https://mlsanity.com/paper/2408.15998/) (2024)
- [MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering](https://mlsanity.com/paper/2212.09662/) (2022)
- [ChartBench: A Benchmark for Complex Visual Reasoning in Charts](https://mlsanity.com/paper/2312.15915/) (2023)
- [ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild](https://mlsanity.com/paper/2407.04172/) (2024)
- [ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning](https://mlsanity.com/paper/2403.09028/) (2024)
- [PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning](https://mlsanity.com/paper/2311.08711/) (2023)
- [ReMI: A Dataset for Reasoning with Multiple Images](https://mlsanity.com/paper/2406.09175/) (2024)
- [Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models](https://mlsanity.com/paper/2310.13127/) (2023)
- [ChartThinker: A Contextual Chain-of-Thought Approach to Optimized Chart Summarization](https://mlsanity.com/paper/2403.11236/) (2024)
- [Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks](https://mlsanity.com/paper/2410.01744/) (2024)
- [Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Training](https://mlsanity.com/paper/2404.14604/) (2024)
- [Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoning](https://mlsanity.com/paper/2406.12050/) (2024)

## Built on by (most cited first, 24 of 24)

- [Baichuan-Omni-1.5 Technical Report](https://mlsanity.com/paper/2501.15368/) (2025)
- [Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2504.21277/) (2025)
- [DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding](https://mlsanity.com/paper/2508.07313/) (2025)
- [MuSciClaims: Multimodal Scientific Claim Verification](https://mlsanity.com/paper/2506.04585/) (2025)
- [ChartGen: Scaling Chart Understanding Via Code-Guided Synthetic Chart Generation](https://mlsanity.com/paper/2507.19492/) (2025)
- [VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs](https://mlsanity.com/paper/2507.13361/) (2025)
- [MageBench: Bridging Large Multimodal Models to Agents](https://mlsanity.com/paper/2412.04531/) (2024)
- [Chart Question Answering from Real-World Analytical Narratives](https://mlsanity.com/paper/2507.01627/) (2025)
- [DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards](https://mlsanity.com/paper/2508.17398/) (2025)
- [ChartAB: A Benchmark for Chart Grounding & Dense Alignment](https://mlsanity.com/paper/2510.26781/) (2025)
- [Chart-RL: Generalized Chart Comprehension via Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2603.06958/) (2026)
- [ChartDiff: A Large-Scale Benchmark for Comprehending Pairs of Charts](https://mlsanity.com/paper/2603.28902/) (2026)
- [GuessBench: Sensemaking Multimodal Creativity in the Wild](https://mlsanity.com/paper/2506.00814/) (2025)
- [ChartAnchor: Chart Grounding with Structural-Semantic Fidelity](https://mlsanity.com/paper/2512.01017/) (2025)
- [Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement](https://mlsanity.com/paper/2602.10138/) (2026)
- [Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge](https://mlsanity.com/paper/2604.22939/) (2026)
- [AnnoBench: A Benchmark for Visualization Annotation Generation](https://mlsanity.com/paper/2607.25911/) (2026)
- [Benchmark It Yourself (BIY): Preparing a Dataset and Benchmarking AI Models for Scatterplot-Related Tasks](https://mlsanity.com/paper/2510.06071/) (2025)
- [Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts](https://mlsanity.com/paper/2604.21344/) (2026)
- [SD-GRPO: Verifiable Segment Decomposition for Long-Form Vision-Language Generation](https://mlsanity.com/paper/2606.09871/) (2026)
- [Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy](https://mlsanity.com/paper/2607.15176/) (2026)
- [ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation](https://mlsanity.com/paper/2608.03464/) (2026)
- [How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures](https://mlsanity.com/paper/2608.13267/) (2026)
- [LegendBench: A Diagnostic Benchmark for Legend Understanding with Counterfactual Interventions](https://mlsanity.com/paper/2609.24172/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated impressive abilities across various tasks, including visual question answering and chart comprehension, yet existing benchmarks for chart-related tasks fall short in capturing the complexity of real-world multi-chart scenarios. Current benchmarks primarily focus on single-chart tasks, neglecting the multi-hop reasoning required to extract and integrate information from multiple charts, which is essential in practical applications. To fill this gap, we introduce MultiChartQA, a benchmark that evaluates MLLMs' capabilities in four key areas: direct question answering, parallel question answering, comparative reasoning, and sequential reasoning. Our evaluation of a wide range of MLLMs reveals significant performance gaps compared to humans. These results highlight the challenges in multi-chart comprehension and the potential of MultiChartQA to drive advancements in this field. Our code and data are available at https://github.com/Zivenzhu/Multi-chart-QA