# SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings

arXiv: https://arxiv.org/abs/2403.17784 · HTML: https://mlsanity.com/paper/2403.17784/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [DVQA: Understanding Data Visualizations via Question Answering](https://mlsanity.com/paper/1801.08163/) (2018)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [Chart-to-Text: A Large-Scale Benchmark for Chart Summarization](https://mlsanity.com/paper/2203.06486/) (2022)
- [UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning](https://mlsanity.com/paper/2305.14761/) (2023)
- [MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering](https://mlsanity.com/paper/2212.09662/) (2022)
- [VisText: A Benchmark for Semantically Rich Chart Captioning](https://mlsanity.com/paper/2307.05356/) (2023)
- [Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model](https://mlsanity.com/paper/2010.09142/) (2020)
- [SciCap: Generating Captions for Scientific Figures](https://mlsanity.com/paper/2110.11624/) (2021)
- [SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs](https://mlsanity.com/paper/2308.03349/) (2023)
- [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022)
- [OpenCQA: Open-ended Question Answering with Charts](https://mlsanity.com/paper/2210.06628/) (2022)
- [SciCap+: A Knowledge Augmented Dataset to Study the Challenges of Scientific Figure Captioning](https://mlsanity.com/paper/2306.03491/) (2023)
- [GPT-4 as an Effective Zero-Shot Evaluator for Scientific Figure Captions](https://mlsanity.com/paper/2310.15405/) (2023)
- [SCITUNE: Aligning Large Language Models with Human-Curated Scientific Multimodal Instructions](https://mlsanity.com/paper/2307.01139/) (2023)
- [LineCap: Line Charts for Data Visualization Captioning Models](https://mlsanity.com/paper/2207.07243/) (2022)
- [Striking a Balance: Reader Takeaways and Preferences when Integrating Text and Charts](https://mlsanity.com/paper/2208.01780/) (2022)
- [Summaries as Captions: Generating Figure Captions for Scientific Documents with Automated Text Summarization](https://mlsanity.com/paper/2302.12324/) (2023)
- [Towards Understanding How Readers Integrate Charts and Captions: A Case Study with Line Charts](https://mlsanity.com/paper/2101.08235/) (2021)
- [InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebooks](https://mlsanity.com/paper/2307.07922/) (2023)
- [ACL-Fig: A Dataset for Scientific Figure Classification](https://mlsanity.com/paper/2301.12293/) (2023)
- [FigCaps-HF: A Figure-to-Caption Generative Framework and Benchmark with Human Feedback](https://mlsanity.com/paper/2307.10867/) (2023)

## Built on by (most cited first, 5 of 5)

- [TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning](https://mlsanity.com/paper/2603.03072/) (2026)
- [Multi-LLM Collaborative Caption Generation in Scientific Documents](https://mlsanity.com/paper/2501.02552/) (2025)
- [Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SciCap Challenge 2023](https://mlsanity.com/paper/2501.19353/) (2025)
- [LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure Profiles](https://mlsanity.com/paper/2506.06561/) (2025)
- [Five Years of SciCap: What We Learned and Future Directions for Scientific Figure Captioning](https://mlsanity.com/paper/2512.21789/) (2025)

## Abstract (arXiv)

Crafting effective captions for figures is important. Readers heavily depend on these captions to grasp the figure's message. However, despite a well-developed set of AI technologies for figures and captions, these have rarely been tested for usefulness in aiding caption writing. This paper introduces SciCapenter, an interactive system that puts together cutting-edge AI technologies for scientific figure captions to aid caption composition. SciCapenter generates a variety of captions for each figure in a scholarly article, providing scores and a comprehensive checklist to assess caption quality across multiple critical aspects, such as helpfulness, OCR mention, key takeaways, and visual properties reference. Users can directly edit captions in SciCapenter, resubmit for revised evaluations, and iteratively refine them. A user study with Ph.D. students indicates that SciCapenter significantly lowers the cognitive load of caption writing. Participants' feedback further offers valuable design insights for future systems aiming to enhance caption writing.