Text to Automata Diagrams: Comparing TikZ Code Generation with Direct Image Synthesis
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Diagrams are widely used in teaching computer science courses. They are useful in subjects such as automata and formal languages, data structures, etc. These diagrams, often drawn by students during exams or assignments, vary in structure, layout, and correctness. This study examines whether current vision-language and large language models can process such diagrams and produce accurate textual and digital representations. In this study, scanned student-drawn diagrams are used as input. Then, textual descriptions are generated from these images using a vision-language model. The descriptions are checked and revised by human reviewers to make them accurate. Both the generated and the revised descriptions are then fed to a large language model to generate TikZ code. The resulting diagrams are compiled and then evaluated against the original scanned diagrams. We found descriptions generated directly from images using vision-language models are often incorrect and human correction can substantially improve the quality of vision language model generated descriptions. This research can help computer science education by paving the way for automated grading and feedback and creating more accessible instructional materials.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 18 references that are in the map. First posted on arXiv 9 Mar 2026.
Builds on
13 of 18 references.
References · 13
- 2025ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generationcited by 92
- 2021SciCap: Generating Captions for Scientific Figurescited by 77
- 2023SciCap+: A Knowledge Augmented Dataset to Study the Challenges of Scientific Figure Captioningcited by 24
- 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 24
- 2024ScImage: How Good Are Multimodal Large Language Models at Scientific Text-to-Image Generation?cited by 13
- 2025Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterizationcited by 9
- 2025A Review on Generative AI For Text-To-Image and Image-To-Image Generation and Implications To Scientific Imagescited by 8
- 2025DiagramEval: Evaluating LLM-Generated Diagrams via Graphscited by 8
- 2025PatentLMM: Large Multimodal Model for Generating Descriptions for Patent Figurescited by 7
- 2025ChartCap: Mitigating Hallucination of Dense Chart Captioningcited by 7
- 2025Chart-HQA: A Benchmark for Hypothetical Question Answering in Chartscited by 6
- 2023A Comparative Analysis of Text-to-Image Generative AI Models in Scientific Contexts: A Case Study on Nuclear Powercited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilities
- 2026SketchJudge: A Diagnostic Benchmark for Grading Hand-drawn Diagrams with Multimodal Large Language Models
- 2026Exploring Agentic Workflows for Generating High Quality Math Visual Aids
- 2026CAGE: Bridging the Accuracy-Aesthetics Gap in Educational Diagrams via Code-Anchored Generative Enhancement
- 2025DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images
- 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward
- 2024Do Vision-Language Models Really Understand Visual Language?
- 2024MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems