mlsanityData to 14 Oct 2026

FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models

arXiv 2507.14823 ↗First posted 20 Jul 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Large vision-language models (LVLMs) have made significant progress in chart understanding. However, financial charts, characterized by complex temporal structures and domain-specific terminology, remain notably underexplored. We introduce FinChart-Bench, the first benchmark specifically focused on real-world financial charts. FinChart-Bench comprises 1,200 financial chart images collected from 2015 to 2024, each annotated with True/False (TF), Multiple Choice (MC), and Question Answering (QA) questions, totaling 7,016 questions. We conduct a comprehensive evaluation of 25 state-of-the-art LVLMs on FinChart-Bench. Our evaluation reveals critical insights: (1) the performance gap between open-source and closed-source models is narrowing, (2) performance degradation occurs in upgraded models within families, (3) many models struggle with instruction following, (4) both advanced models show significant limitations in spatial reasoning abilities, and (5) current LVLMs are not reliable enough to serve as automated evaluators. These findings highlight important limitations in current LVLM capabilities for financial chart understanding. The FinChart-Bench dataset is available at https://huggingface.co/datasets/Tizzzzy/FinChart-Bench.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 53 references that are in the map. First posted on arXiv 20 Jul 2025.

Builds on

21 of 53 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering11 May
  2. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning19 Feb
  3. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning4 Jan
  4. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Charts26 Dec
  5. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generation27 Nov
  6. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning15 Nov
  7. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning24 May

Other references · 14

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  7. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  8. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 169
  9. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Modelscited by 139
  10. 2021SciCap: Generating Captions for Scientific Figurescited by 77
  11. 2023SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphscited by 68
  12. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 68
2 more
  1. 2024Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoningcited by 54
  2. 2025Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainabilitycited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026FinTradeBench: A Financial Reasoning Benchmark for LLMscited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2025Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning?
  2. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMs
  3. 2026Confidence Estimation for Financial Vision-Language Models in Chart and Document Understanding
  4. 2025Losing the Plot: How VLM responses degrade on imperfect charts
  5. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
  6. 2025InterChart: Benchmarking Visual Reasoning Across Decomposed and Distributed Chart Information
  7. 2026FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation
  8. 2025POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering