mlsanityData to 14 Oct 2026

Unlocking UML Class Diagram Understanding in Vision Language Models

arXiv 2605.11634 ↗First posted 12 May 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Although Vision Language Models (VLMs) have seen tremendous progress across all kinds of use cases, they still fall behind in answering questions regard-ing diagrams compared to photos. Although progress has been made in the area of bar charts, line charts and other diagrams like that there is still few research concerned with other types of diagrams, e.g. in the computer science domain. Our work presents a benchmark for visual question answering based on UML class diagrams which is both challenging and manageable. We further construct a large-scale training dataset with 16.000 image-question-answer triples and show that a LoRA-based finetune easily outperforms Qwen 3.5 27B, which is a recent and well-performing VLM in many other benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 20 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

15 of 20 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Limits and Gains of Test-Time Scaling in Vision-Language Reasoning11 Dec
  2. 2025VLM@school -- Evaluation of AI image understanding on German middle school knowledge13 Jun

Other references · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  6. 2025ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answeringcited by 108
  7. 2024Unraveling the Truth: Do VLMs really Understand Charts? A Deep Dive into Consistency and Robustnesscited by 17
  8. 2024LLaVA-Gemma: Accelerating Multimodal Foundation Models with a Compact Language Modelcited by 14
  9. 2024From Image to UML: First Results of Image Based UML Diagram Generation Using LLMscited by 11
  10. 2025MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curationcited by 4
  11. 2025Structured Extraction from Business Process Diagrams Using Vision-Language Modelscited by 3
  12. 2025NOMAD: A Multi-Agent LLM System for UML Class Diagram Generation from Natural Language Requirementscited by 2
1 more
  1. 2025Measuring Visual Understanding in Telecom domain: Performance Metrics for Image-to-UML conversion using VLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types
  2. 2026DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams
  3. 2024Can Visual Language Models Replace OCR-Based Visual Question Answering Pipelines in Production? A Case Study in Retail
  4. 2025Probing the Visualization Literacy of Vision Language Models: the Good, the Bad, and the Ugly
  5. 2025ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
  6. 2024VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation
  7. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  8. 2024Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy