mlsanityData to 7 Oct 2026

Chart Question Answering: State of the Art and Future Directions

arXiv 2205.03966 ↗First posted 8 May 2022cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

Information visualizations such as bar charts and line charts are very common for analyzing data and discovering critical insights. Often people analyze charts to answer questions that they have in mind. Answering such questions can be challenging as they often require a significant amount of perceptual and cognitive effort. Chart Question Answering (CQA) systems typically take a chart and a natural language question as input and automatically generate the answer to facilitate visual data analysis. Over the last few years, there has been a growing body of literature on the task of CQA. In this survey, we systematically review the current state-of-the-art research focusing on the problem of chart question answering. We provide a taxonomy by identifying several important dimensions of the problem domain including possible inputs and outputs of the task and discuss the advantages and limitations of proposed solutions. We then summarize various evaluation techniques used in the surveyed papers. Finally, we outline the open challenges and future research opportunities related to chart question answering.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 50 of its 114 references that are in the map. First posted on arXiv 8 May 2022.

Builds on

50 of 114 references.

References · 50

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2017Mask R-CNNcited by 7,259
  4. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  7. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  8. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  9. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  10. 2018CoQA: A Conversational Question Answering Challengecited by 674
  11. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  12. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
38 more
  1. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  2. 2016Visual Dialogcited by 377
  3. 2017A simple neural network module for relational reasoningcited by 345
  4. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  5. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  6. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  7. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  8. 2021A Survey of Visual Transformerscited by 117
  9. 2020Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Modelcited by 77
  10. 2020NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queriescited by 73
  11. 2019BERT with History Answer Embedding for Conversational Question Answeringcited by 67
  12. 2021Towards Natural Language Interfaces for Data Visualization: A Surveycited by 58
  13. 2019LEAF-QA: Locate, Encode & Attend for Figure Question Answeringcited by 53
  14. 2021Complex Knowledge Base Question Answering: A Surveycited by 52
  15. 2018VizML: A Machine Learning Approach to Visualization Recommendationcited by 38
  16. 2021Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Contentcited by 38
  17. 2019Text-to-Viz: Automatic Generation of Infographics from Proportion-Related Natural Language Statementscited by 31
  18. 2021Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizationscited by 31
  19. 2020Calliope: Automatic Visual Data Story Generation from a Spreadsheetcited by 30
  20. 2019Answering Questions about Data Visualizations using Efficient Bimodal Fusioncited by 26
  21. 2019FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow Systemcited by 25
  22. 2019Data Extraction from Charts via Single Deep Neural Networkcited by 24
  23. 2019Towards Automated Infographic Design: Deep Learning-based Auto-Extraction of Extensible Timelinecited by 20
  24. 2017Beagle: Automated Extraction and Interpretation of Visualizations from the Webcited by 19
  25. 2020VIS30K: A Collection of Figures and Tables from IEEE Visualization Conference Publicationscited by 13
  26. 2020InChorus: Designing Consistent Multimodal Interactions for Data Visualization on Tablet Devicescited by 12
  27. 2019Searching the Visual Style and Structure of D3 Visualizationscited by 11
  28. 2021ChartStory: Automated Partitioning, Layout, and Captioning of Charts into Comic-Style Narrativescited by 10
  29. 2016Converting Basic D3 Charts into Reusable Style Templatescited by 7
  30. 2020Composition and Configuration Patterns in Multiple-View Visualizationscited by 7
  31. 2020PlotThread: Creating Expressive Storyline Visualizations using Reinforcement Learningcited by 7
  32. 2017Coupling Story to Visualization: Using Textual Analysis as a Bridge Between Data and Interpretationcited by 6
  33. 2018FlipDial: A Generative Model for Two-Way Visual Dialoguecited by 6
  34. 2018FigureNet: A Deep Learning model for Question-Answering on Scientific Plotscited by 6
  35. 2021Deep Colormap Extraction from Visualizationscited by 6
  36. 2020Touch? Speech? or Touch and Speech? Investigating Multimodal Interaction for Visual Network Exploration and Analysiscited by 5
  37. 2021An Evaluation-Focused Framework for Visualization Recommendation Algorithmscited by 3
  38. 2021VizAI : Selecting Accurate Visualizations of Numerical Datacited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 9; 2025: 5; 2026: 6.

26 papers.

Sort:

By area:

  1. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
  2. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuningcited by 79
  3. 2024ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wildcited by 79
  4. 2022OpenCQA: Open-ended Question Answering with Chartscited by 60
  5. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  6. 2023Multimodal Automated Fact-Checking: A Surveycited by 51
  7. 2023ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summariescited by 30
  8. 2024How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?cited by 27
  9. 2024Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuningcited by 27
  10. 2024Generative AI for Visualization: State of the Art and Future Directionscited by 23
  11. 2025BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuningcited by 13
  12. 2023DOMINO: A Dual-System for Multi-step Visual Language Reasoningcited by 12
14 more
  1. 2024mChartQA: A universal benchmark for multimodal Chart Question Answer based on Vision-Language Alignment and Reasoningcited by 9
  2. 2025Probing the Visualization Literacy of Vision Language Models: the Good, the Bad, and the Uglycited by 9
  3. 2026QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understandingcited by 9
  4. 2025See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMscited by 7
  5. 2025Charts-of-Thought: Enhancing LLM Visualization Literacy Through Structured Data Extractioncited by 6
  6. 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directionscited by 5
  7. 2025The Perils of Chart Deception: How Misleading Visualizations Affect Vision-Language Modelscited by 5
  8. 2023LineFormer: Rethinking Line Chart Data Extraction as Instance Segmentationcited by 4
  9. 2024LLM-Assisted Visual Analytics: Opportunities and Challengescited by 2
  10. 2026Making Multimodal LLMs Reliable Chart Data Extractors: A Benchmark and Training Frameworkcited by 2
  11. 2026Generating Statistical Charts with Validation-Driven LLM Workflowscited by 1
  12. 2026Chart Deep Research in LVLMs via Parallel Relative Policy Optimizationcited by 0
  13. 2026Query2Diagram: Answering Developer Queries with UML Diagramscited by 0
  14. 2026Chart Deception in Vision-Language Models: From Vulnerability to Mitigationcited by 0