mlsanityData to 7 Oct 2026

OpenCQA: Open-ended Question Answering with Charts

arXiv 2210.06628 ↗First posted 12 Oct 2022cs.LG cs.CLcited by 60

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Charts are very popular to analyze data and convey important insights. People often analyze visualizations to answer open-ended questions that require explanatory answers. Answering such questions are often difficult and time-consuming as it requires a lot of cognitive and perceptual efforts. To address this challenge, we introduce a new task called OpenCQA, where the goal is to answer an open-ended question about a chart with descriptive texts. We present the annotation process and an in-depth analysis of our dataset. We implement and evaluate a set of baselines under three practical settings. In the first setting, a chart and the accompanying article is provided as input to the model. The second setting provides only the relevant paragraph(s) to the chart instead of the entire article, whereas the third setting requires the model to generate an answer solely based on the chart. Our analysis of the results show that the top performing models generally produce fluent and coherent text while they struggle to perform complex logical and arithmetic reasoning.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

47 of 69 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2017Mask R-CNNcited by 7,259
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  11. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,837
35 more
  1. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  3. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  4. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  5. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  6. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  7. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  8. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  9. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  10. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 363
  11. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  12. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  13. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  14. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  15. 2017Challenges in Data-to-Document Generationcited by 226
  16. 2022Memorizing Transformerscited by 219
  17. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 177
  18. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  19. 2016Graph-Structured Representations for Visual Question Answeringcited by 123
  20. 2020Logical Natural Language Generation from Open-Domain Tablescited by 90
  21. 2020Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Modelcited by 77
  22. 2019LEAF-QA: Locate, Encode & Attend for Figure Question Answeringcited by 53
  23. 2020Pre-Training Transformers as Energy-Based Cloze Modelscited by 52
  24. 2016Dynamic Coattention Networks For Question Answeringcited by 49
  25. 2020Logic2Text: High-Fidelity Natural Language Generation from Logical Formscited by 45
  26. 2017Diversity driven Attention Model for Query-based Abstractive Summarizationcited by 42
  27. 2018Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Modelscited by 39
  28. 2015What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignmentcited by 31
  29. 2022Chart Question Answering: State of the Art and Future Directionscited by 26
  30. 2021Focused Attention Improves Document-Grounded Generationcited by 24
  31. 2020WSL-DS: Weakly Supervised Learning with Distant Supervision for Query Focused Multi-Document Abstractive Summarizationcited by 23
  32. 2019Figure Captioning with Reasoning and Sequence-Level Trainingcited by 21
  33. 2021Data Augmentation for Abstractive Query-Focused Multi-Document Summarizationcited by 18
  34. 2016Reference-Aware Language Modelscited by 8
  35. 2019BERTQA -- Attention on Steroidscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 22; 2025: 21; 2026: 10.

60 papers.

Sort:

By area:

  1. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
  2. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
  3. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 127
  4. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Chartscited by 116
  5. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 105
  6. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuningcited by 79
  7. 2024ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wildcited by 79
  8. 2024ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoningcited by 68
  9. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  10. 2024OneChart: Purify the Chart Structural Extraction via One Auxiliary Tokencited by 39
  11. 2023ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summariescited by 30
  12. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
38 more
  1. 2024Distill Visual Chart Reasoning Ability from LLMs to MLLMscited by 30
  2. 2024DataNarrative: Automated Data-Driven Storytelling with Visualizations and Textscited by 25
  3. 2023ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rulescited by 24
  4. 2023OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamesecited by 20
  5. 2024Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answeringcited by 20
  6. 2025Effective Training Data Synthesis for Improving MLLM Chart Understandingcited by 20
  7. 2024Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustnesscited by 17
  8. 2025InfoChartQA: A Benchmark for Multimodal Question Answering on Infographic Chartscited by 16
  9. 2024What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biasescited by 13
  10. 2024On Pre-training of Multimodal Language Models Customized for Chart Understandingcited by 13
  11. 2025BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuningcited by 13
  12. 2023Do LLMs Work on Charts? Designing Few-Shot Prompts for Chart Question Answering and Summarizationcited by 10
  13. 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinementcited by 10
  14. 2025ChartM3: Benchmarking Chart Editing with Multimodal Instructionscited by 9
  15. 2024ViOCRVQA: Novel Benchmark Dataset and Vision Reader for Visual Question Answering by Understanding Vietnamese Text in Imagescited by 8
  16. 2025WikiMixQA: A Multimodal Benchmark for Question Answering over Tables and Chartscited by 8
  17. 2024SIMPLOT: Enhancing Chart Question Answering by Distilling Essentialscited by 7
  18. 2024Understanding Graphical Perception in Data Visualization through Zero-shot Prompting of Vision-Language Modelscited by 6
  19. 2025ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answeringcited by 6
  20. 2026Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projectioncited by 6
  21. 2024SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratingscited by 5
  22. 2024Advancing Chart Question Answering with Robust Chart Component Recognitioncited by 5
  23. 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directionscited by 5
  24. 2024Enhancing Question Answering on Charts Through Effective Pre-training Taskscited by 4
  25. 2024ChartAdapter: Large Vision-Language Model for Chart Summarizationcited by 4
  26. 2023PAT: Parallel Attention Transformer for Visual Question Answering in Vietnamesecited by 3
  27. 2025POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answeringcited by 3
  28. 2025DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboardscited by 3
  29. 2025ChartAB: A Benchmark for Chart Grounding & Dense Alignmentcited by 3
  30. 2025In-Depth and In-Breadth: Pre-training Multimodal Language Models Customized for Comprehensive Chart Understandingcited by 2
  31. 2025From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Textcited by 2
  32. 2025ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinementcited by 2
  33. 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualizationcited by 2
  34. 2025Data Metabolism: An Efficient Data Design Schema For Vision Language Modelcited by 1
  35. 2025CHAOS: Chart Analysis with Outlier Samplescited by 1
  36. 2025MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMscited by 1
  37. 2025ChartAnchor: Chart Grounding with Structural-Semantic Fidelitycited by 1
  38. 2026DATAREEL: Automated Data-Driven Video Story Generation with Animationscited by 1

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DCQA: Document-Level Chart Question Answering towards Complex Reasoning and Common-Sense Understanding
  2. 2025Chart Question Answering from Real-World Analytical Narratives
  3. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering
  4. 2024GoT-CQA: Graph-of-Thought Guided Compositional Reasoning for Chart Question Answering
  5. 2026Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts
  6. 2024Charting the Future: Using Chart Question-Answering for Scalable Evaluation of LLM-Driven Data Visualizations
  7. 2024Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA
  8. 2026What Lies Beneath: A Call for Distribution-based Visual Question & Answer Datasets