mlsanityData to 14 Oct 2026

ChartBench: A Benchmark for Complex Visual Reasoning in Charts

arXiv 2312.15915 ↗First posted 26 Dec 2023cs.CVcited by 120

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have shown impressive capabilities in image understanding and generation. However, current benchmarks fail to accurately evaluate the chart comprehension of MLLMs due to limited chart types and inappropriate metrics. To address this, we propose ChartBench, a comprehensive benchmark designed to assess chart comprehension and data reliability through complex visual reasoning. ChartBench includes 42 categories, 66.6k charts, and 600k question-answer pairs. Notably, many charts lack data point annotations, which requires MLLMs to derive values similar to human understanding by leveraging inherent chart elements such as color, legends, and coordinate systems. We also design an enhanced evaluation metric, Acc+, to evaluate MLLMs without extensive manual or costly LLM-based evaluations. Furthermore, we propose two baselines based on the chain of thought and supervised fine-tuning to improve model performance on unannotated charts. Extensive experimental evaluations of 18 open-sourced and 3 proprietary MLLMs reveal their limitations in chart comprehension and offer valuable insights for further research. Code and dataset are publicly available at https://chartbench.github.io.

Key facts. Cited by 120 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 63 of its 85 references that are in the map. First posted on arXiv 26 Dec 2023.

Builds on

63 of 85 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023DOMINO: A Dual-System for Multi-step Visual Language Reasoning4 Oct
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  3. 2022DePlot: One-shot visual language reasoning by plot-to-table translation20 Dec
  4. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering19 Dec
  5. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  10. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  11. 2023Visual Instruction Tuningcited by 7,869
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
38 more
  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  5. 2023Qwen Technical Reportcited by 3,344
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  11. 2015VQA: Visual Question Answeringcited by 2,506
  12. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  13. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  14. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  15. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  16. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  17. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  18. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  19. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  20. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  21. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  22. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  23. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  24. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  25. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  26. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  27. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  28. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  29. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  30. 2023GPTScore: Evaluate as You Desirecited by 361
  31. 2019PlotQA: Reasoning over Scientific Plotscited by 321
  32. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  33. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 286
  34. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  35. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 191
  36. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 169
  37. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 163
  38. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 25; 2025: 60; 2026: 35.

120 papers.

Sort:

By area:

  1. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  2. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  3. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
  4. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 128
  5. 2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 123
  6. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  7. 2024TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domainscited by 69
  8. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
  9. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 53
  10. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answeringcited by 50
  11. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  12. 2024ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understandingcited by 46
38 more
  1. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 44
  2. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  3. 2025METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scalingcited by 34
  4. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Modelscited by 32
  5. 2024Distill Visual Chart Reasoning Ability from LLMs to MLLMscited by 31
  6. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
  7. 2024FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowchartscited by 28
  8. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  9. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  10. 2024MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problemscited by 24
  11. 2025Effective Training Data Synthesis for Improving MLLM Chart Understandingcited by 20
  12. 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performancecited by 19
  13. 2025SridBench: Benchmark of Scientific Research Illustration Drawing of Image Generation Modelcited by 19
  14. 2024Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustnesscited by 17
  15. 2025InfoChartQA: A Benchmark for Multimodal Question Answering on Infographic Chartscited by 17
  16. 2026IGenBench: Benchmarking the Reliability of Text-to-Infographic Generationcited by 17
  17. 2025Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answeringcited by 16
  18. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  19. 2024On Pre-training of Multimodal Language Models Customized for Chart Understandingcited by 13
  20. 2025ChartReasoner: Code-Driven Modality Bridging for Long-Chain Reasoning in Chart Question Answeringcited by 13
  21. 2025Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curationcited by 13
  22. 2025Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skippingcited by 12
  23. 2025Data-to-Dashboard: Multi-Agent LLM Framework for Insightful Visualization in Enterprise Analyticscited by 11
  24. 2025On the Perception Bottleneck of VLMs for Chart Understandingcited by 10
  25. 2025VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstructioncited by 10
  26. 2026ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratchcited by 10
  27. 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Reviewcited by 9
  28. 2025Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluatorcited by 9
  29. 2025MuSciClaims: Multimodal Scientific Claim Verificationcited by 8
  30. 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesiscited by 8
  31. 2025ChartGen: Scaling Chart Understanding Via Code-Guided Synthetic Chart Generationcited by 8
  32. 2026ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understandingcited by 8
  33. 2025See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMscited by 7
  34. 2025VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanismcited by 7
  35. 2025ChartCap: Mitigating Hallucination of Dense Chart Captioningcited by 7
  36. 2025Chimera: Diagnosing Shortcut Learning in Visual-Language Understandingcited by 7
  37. 2025ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answeringcited by 7
  38. 2025LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understandingcited by 7

50 most cited of 120 All 120 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025EncQA: Benchmarking Vision-Language Models on Visual Encodings for Charts
  2. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning
  3. 2025ChartSketcher: Reasoning with Multimodal Feedback and Reflection for Chart Understanding
  4. 2025InterChart: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information
  5. 2025ChartM3: A Multi-Stage Code-Driven Pipeline for Constructing Multi-Dimensional and Multi-Step Visual Reasoning Data in Chart Comprehension
  6. 2025START: Spatial and Textual Learning for Chart Understanding
  7. 2025Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning?
  8. 2026ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models