mlsanityData to 7 Oct 2026

Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model

arXiv 2010.09142 ↗First posted 18 Oct 2020cs.CL cs.AIcited by 77

Not on the map. Why?

Cited by 28 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Information visualizations such as bar charts and line charts are very popular for exploring data and communicating insights. Interpreting and making sense of such visualizations can be challenging for some people, such as those who are visually impaired or have low visualization literacy. In this work, we introduce a new dataset and present a neural model for automatically generating natural language summaries for charts. The generated summaries provide an interpretation of the chart and convey the key insights found within that chart. Our neural model is developed by extending the state-of-the-art model for the data-to-text generation task, which utilizes a transformer-based encoder-decoder architecture. We found that our approach outperforms the base model on a content selection metric by a wide margin (55.42% vs. 8.49%) and generates more informative, concise, and coherent summaries.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 7; 2023: 11; 2024: 25; 2025: 20; 2026: 8.

77 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2024What matters when building vision-language models?cited by 321
  4. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  5. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  6. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
  7. 2024ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoningcited by 127
  8. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  9. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 124
  10. 2023VisText: A Benchmark for Semantically Rich Chart Captioningcited by 111
  11. 2021SciCap: Generating Captions for Scientific Figurescited by 76
  12. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 75
38 more
  1. 2024ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoningcited by 68
  2. 2022OpenCQA: Open-ended Question Answering with Chartscited by 60
  3. 2024From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Modelscited by 52
  4. 2024TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learningcited by 51
  5. 2025FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocitiescited by 46
  6. 2024OneChart: Purify the Chart Structural Extraction via One Auxiliary Tokencited by 39
  7. 2021Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Contentcited by 38
  8. 2024SOLO: A Single Transformer for Scalable Vision-Language Modelingcited by 35
  9. 2022Language Models as Inductive Reasonerscited by 32
  10. 2023ChartSumm: A Comprehensive Benchmark for Automatic Chart Summarization of Long and Short Summariescited by 30
  11. 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
  12. 2022Chart Question Answering: State of the Art and Future Directionscited by 26
  13. 2023ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rulescited by 24
  14. 2023DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articlescited by 24
  15. 2025Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligencecited by 20
  16. 2021Underreporting of errors in NLG output, and what to do about itcited by 19
  17. 2021Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Paperscited by 18
  18. 2022LineCap: Line Charts for Data Visualization Captioning Modelscited by 17
  19. 2024Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generationcited by 16
  20. 2023Enhanced Chart Understanding in Vision and Language Task via Cross-modal Pre-training on Plot Table Pairscited by 15
  21. 2025LLM/Agent-as-Data-Analyst: A Surveycited by 15
  22. 2023Summaries as Captions: Generating Figure Captions for Scientific Documents with Automated Text Summarizationcited by 14
  23. 2021AutoChart: A Dataset for Chart-to-Text Generation Taskcited by 12
  24. 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
  25. 2022TaTa: A Multilingual Table-to-Text Dataset for African Languagescited by 10
  26. 2024OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understandingcited by 10
  27. 2024DOGR: Towards Versatile Visual Document Grounding and Referringcited by 10
  28. 2025ChartInsighter: An Approach for Mitigating Hallucination in Time-series Chart Summary Generation with A Benchmark Datasetcited by 9
  29. 2025MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuningcited by 9
  30. 2025Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Reviewcited by 8
  31. 2023FigCaps-HF: A Figure-to-Caption Generative Framework and Benchmark with Human Feedbackcited by 7
  32. 2025ChatVis: Large Language Model Agent for Generating Scientific Visualizationscited by 7
  33. 2023Tackling Hallucinations in Neural Chart Summarizationcited by 6
  34. 2024Data Analysis in the Era of Generative AIcited by 6
  35. 2021Text2Chart: A Multi-Staged Chart Generator from Natural Language Textcited by 5
  36. 2024SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratingscited by 5
  37. 2024Faithful Chart Summarization with ChaTS-Picited by 5
  38. 2024Natural Language Generation for Visualizations: State of the Art, Challenges and Future Directionscited by 5

50 most cited of 77 All 77 (JSON)