# Assessing GPT-4-Vision's Capabilities in UML-Based Code Generation

arXiv: https://arxiv.org/abs/2404.14370 · HTML: https://mlsanity.com/paper/2404.14370/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 12 references that are in the map.
- First posted on arXiv 22 Apr 2024.

## Builds on (most cited first, 2 of 2)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LLM-based Control Code Generation using Image Recognition](https://mlsanity.com/paper/2311.10401/) (2023)

## Built on by (most cited first, 4 of 4)

- [An Eye for an AI: Evaluating GPT-4o's Visual Perception Skills and Geometric Reasoning Skills Using Computer Graphics Questions](https://mlsanity.com/paper/2410.16991/) (2024)
- [Leveraging ChatGPT for Sponsored Ad Detection and Keyword Extraction in YouTube Videos](https://mlsanity.com/paper/2502.15102/) (2025)
- [Data Dependency-Aware Code Generation from Enhanced UML Sequence Diagrams](https://mlsanity.com/paper/2508.03379/) (2025)
- [Prior Bias in Vision Language Models on UML Diagram Interpretation](https://mlsanity.com/paper/2607.02853/) (2026)

## Abstract (arXiv)

The emergence of advanced neural networks has opened up new ways in automated code generation from conceptual models, promising to enhance software development processes. This paper presents a preliminary evaluation of GPT-4-Vision, a state-of-the-art deep learning model, and its capabilities in transforming Unified Modeling Language (UML) class diagrams into fully operating Java class files. In our study, we used exported images of 18 class diagrams comprising 10 single-class and 8 multi-class diagrams. We used 3 different prompts for each input, and we manually evaluated the results. We created a scoring system in which we scored the occurrence of elements found in the diagram within the source code. On average, the model was able to generate source code for 88% of the elements shown in the diagrams. Our results indicate that GPT-4-Vision exhibits proficiency in handling single-class UML diagrams, successfully transforming them into syntactically correct class files. However, for multi-class UML diagrams, the model's performance is weaker compared to single-class diagrams. In summary, further investigations are necessary to exploit the model's potential completely.