# Multi-modal Summarization in Model-Based Engineering: Automotive Software Development Case Study

arXiv: https://arxiv.org/abs/2503.04506 · HTML: https://mlsanity.com/paper/2503.04506/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Emu3: Next-Token Prediction is All You Need](https://mlsanity.com/paper/2409.18869/) (2024)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge](https://mlsanity.com/paper/2206.01718/) (2022)
- [MM-LLMs: Recent Advances in MultiModal Large Language Models](https://mlsanity.com/paper/2401.13601/) (2024)
- [Pixtral 12B](https://mlsanity.com/paper/2410.07073/) (2024)
- [xGen-MM (BLIP-3): A Family of Open Large Multimodal Models](https://mlsanity.com/paper/2408.08872/) (2024)
- [Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization](https://mlsanity.com/paper/2309.04669/) (2023)
- [AI2D-RST: A multimodal corpus of 1000 primary school science diagrams](https://mlsanity.com/paper/1912.03879/) (2019)
- [A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks](https://mlsanity.com/paper/2411.06284/) (2024)
- [A Survey on Multi-modal Summarization](https://mlsanity.com/paper/2109.05199/) (2021)
- [Can Vision-Language Models be a Good Guesser? Exploring VLMs for Times and Location Reasoning](https://mlsanity.com/paper/2307.06166/) (2023)
- [Enhancing Multimodal Understanding with CLIP-Based Image-to-Text Transformation](https://mlsanity.com/paper/2401.06167/) (2024)
- [Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning](https://mlsanity.com/paper/2405.20834/) (2024)
- [From Image to UML: First Results of Image Based UML Diagram Generation Using LLMs](https://mlsanity.com/paper/2404.11376/) (2024)
- [Synergy of Large Language Model and Model Driven Engineering for Automated Development of Centralized Vehicular Systems](https://mlsanity.com/paper/2404.05508/) (2024)
- [XTQA: Span-Level Explanations of the Textbook Question Answering](https://mlsanity.com/paper/2011.12662/) (2020)
- [The importance of visual modelling languages in generative software engineering](https://mlsanity.com/paper/2411.17976/) (2024)

## Built on by (most cited first, 6 of 6)

- [Survey of GenAI for Automotive Software Development: From Requirements to Executable Code](https://mlsanity.com/paper/2507.15025/) (2025)
- [GenAI for Automotive Software Development: From Requirements to Wheels](https://mlsanity.com/paper/2507.18223/) (2025)
- [LLM-based Iterative Approach to Metamodeling in Automotive](https://mlsanity.com/paper/2503.05449/) (2025)
- [GenAI-based test case generation and execution in SDV platform](https://mlsanity.com/paper/2509.05112/) (2025)
- [LLM-Empowered Event-Chain Driven Code Generation for ADAS in SDV systems](https://mlsanity.com/paper/2511.21877/) (2025)
- [LLM-Based Approach for Enhancing Maintainability of Automotive Architectures](https://mlsanity.com/paper/2509.12798/) (2025)

## Abstract (arXiv)

Multimodal summarization integrating information from diverse data modalities presents a promising solution to aid the understanding of information within various processes. However, the application and advantages of multimodal summarization have not received much attention in model-based engineering (MBE), where it has become a cornerstone in the design and development of complex systems, leveraging formal models to improve understanding, validation and automation throughout the engineering lifecycle. UML and EMF diagrams in model-based engineering contain a large amount of multimodal information and intricate relational data. Hence, our study explores the application of multimodal large language models within the domain of model-based engineering to evaluate their capacity for understanding and identifying relationships, features, and functionalities embedded in UML and EMF diagrams. We aim to demonstrate the transformative potential benefits and limitations of multimodal summarization in improving productivity and accuracy in MBE practices. The proposed approach is evaluated within the context of automotive software development, while many promising state-of-art models were taken into account.