SCAFFOLD: A Large-Scale Structured Dataset of Computer Science Research Figures with Diagram QA and Chain-of-Thought Reasoning Traces
On the map: LLM reasoning → Knowledge- and tool-augmented reasoning
Abstract (arXiv)
Computer science papers rely heavily on diagrams: architecture drawings, system flowcharts, and pipeline schematics that often carry more information than the text around them. There is currently no public dataset that pairs this specific kind of figure with captions, context, questions, answers, and step-by-step reasoning, which is exactly what is needed to train a vision-language model to understand them. We present SCAFFOLDfootnotehttps://github.com/theranjitraut/scaffold, a large-scale structured dataset of computer science research figures with diagram QA and Chain-of-Thought reasoning traces. This dataset consists of (image, caption, context, question-answer, chain-of-thought) tuples from arXiv computer science papers prepared using layout detection and PDF parsing, with an AI-assisted question-generation step. The resulting large-sized SCAFFOLD-157K dataset spans 3,058 papers with 29,887 figures (157,387 pairs), a medium-sized SCAFFOLD-37K dataset (36,797 pairs), and a small-sized SCAFFOLD-12K dataset (12,000 pairs). We used SCAFFOLD-12K for baseline experiments on Qwen2.5-VL-3B-Instruct.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 9 of its 12 references that are in the map. First posted on arXiv 20 Aug 2026.
Builds on
9 of 12 references.
References · 9
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2018Datasheets for Datasetscited by 1,386
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
- 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
- 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
- 2023SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphscited by 68
- 2018Extracting Scientific Figures with Distantly Supervised Neural Networkscited by 36
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SCP-116K: A High-Quality Problem-Solution Dataset and a Generalized Pipeline for Automated Extraction in the Higher Education Science Domain
- 2026Visual Graph Scaffolds for Structural Reasoning in Large Language Models
- 2026From Terminology to Diagrams: Visual-Instruction Generation for Scientific Diagram Understanding
- 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers
- 2025Reasoning Scaffolding: Distilling the Flow of Thought from LLMs
- 2026DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams
- 2026DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams
- 2026Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams