mlsanityData to 7 Oct 2026

SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings

arXiv 2403.17784 ↗First posted 26 Mar 2024cs.HC cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Crafting effective captions for figures is important. Readers heavily depend on these captions to grasp the figure's message. However, despite a well-developed set of AI technologies for figures and captions, these have rarely been tested for usefulness in aiding caption writing. This paper introduces SciCapenter, an interactive system that puts together cutting-edge AI technologies for scientific figure captions to aid caption composition. SciCapenter generates a variety of captions for each figure in a scholarly article, providing scores and a comprehensive checklist to assess caption quality across multiple critical aspects, such as helpfulness, OCR mention, key takeaways, and visual properties reference. Users can directly edit captions in SciCapenter, resubmit for revised evaluations, and iteratively refine them. A user study with Ph.D. students indicates that SciCapenter significantly lowers the cognitive load of caption writing. Participants' feedback further offers valuable design insights for future systems aiming to enhance caption writing.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 24 of its 54 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

24 of 54 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  3. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  4. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  5. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  6. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 166
  7. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 158
  8. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 132
  9. 2023VisText: A Benchmark for Semantically Rich Chart Captioningcited by 111
  10. 2020Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Modelcited by 77
  11. 2021SciCap: Generating Captions for Scientific Figurescited by 76
  12. 2022Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderationcited by 66
12 more
  1. 2023SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphscited by 66
  2. 2022OpenCQA: Open-ended Question Answering with Chartscited by 60
  3. 2023SciCap+: A Knowledge Augmented Dataset to Study the Challenges of Scientific Figure Captioningcited by 24
  4. 2023GPT-4 as an Effective Zero-Shot Evaluator for Scientific Figure Captionscited by 21
  5. 2023SCITUNE: Aligning Large Language Models with Human-Curated Scientific Multimodal Instructionscited by 19
  6. 2022LineCap: Line Charts for Data Visualization Captioning Modelscited by 17
  7. 2022Striking a Balance: Reader Takeaways and Preferences when Integrating Text and Chartscited by 17
  8. 2023Summaries as Captions: Generating Figure Captions for Scientific Documents with Automated Text Summarizationcited by 14
  9. 2021Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Chartscited by 12
  10. 2023InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebookscited by 11
  11. 2023ACL-Fig: A Dataset for Scientific Figure Classificationcited by 10
  12. 2023FigCaps-HF: A Figure-to-Caption Generative Framework and Benchmark with Human Feedbackcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Multi-LLM Collaborative Caption Generation in Scientific Documentscited by 6
  2. 2025Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SciCap Challenge 2023cited by 4
  3. 2026TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learningcited by 3
  4. 2025LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure Profilescited by 2
  5. 2025Five Years of SciCap: What We Learned and Future Directions for Scientific Figure Captioningcited by 0