# Safety Case Patterns for VLA-based driving systems: Insights from SimLingo

arXiv: https://arxiv.org/abs/2603.16013 · HTML: https://mlsanity.com/paper/2603.16013/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving.
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- End-to-end autonomous driving / [Reasoning VLA and VLM driving policies](https://mlsanity.com/driving/reasoning-vla-and-vlm-driving-policies/)

## Builds on (most cited first, 26 of 26)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [LMDrive: Closed-Loop End-to-End Driving with Large Language Models](https://mlsanity.com/paper/2312.07488/) (2023)
- [AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.13757/) (2025)
- [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024)
- [OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model](https://mlsanity.com/paper/2503.23463/) (2025)
- [SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment](https://mlsanity.com/paper/2503.09594/) (2025)
- [A Survey on Vision-Language-Action Models: An Action Tokenization Perspective](https://mlsanity.com/paper/2507.01925/) (2025)
- [Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges](https://mlsanity.com/paper/2505.04769/) (2025)
- [A Survey on Vision-Language-Action Models for Autonomous Driving](https://mlsanity.com/paper/2506.24044/) (2025)
- [DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Driving](https://mlsanity.com/paper/2505.19381/) (2025)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [Guidance on the Assurance of Machine Learning in Autonomous Systems (AMLAS)](https://mlsanity.com/paper/2102.01564/) (2021)
- [Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future](https://mlsanity.com/paper/2512.16760/) (2025)
- [LangCoop: Collaborative Driving with Language](https://mlsanity.com/paper/2504.13406/) (2025)
- [Ergo, SMIRK is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System](https://mlsanity.com/paper/2204.07874/) (2022)
- [The BIG Argument for AI Safety Cases](https://mlsanity.com/paper/2503.11705/) (2025)
- [A Principles-based Ethics Assurance Argument Pattern for AI and Autonomous Systems](https://mlsanity.com/paper/2203.15370/) (2022)
- [CoDefeater: Using LLMs To Find Defeaters in Assurance Cases](https://mlsanity.com/paper/2407.13717/) (2024)
- [Model Based System Assurance Using the Structured Assurance Case Metamodel](https://mlsanity.com/paper/1905.02427/) (2019)
- [Security Assurance Cases -- State of the Art of an Emerging Approach](https://mlsanity.com/paper/2003.14151/) (2020)
- [Automatic Instantiation of Assurance Cases from Patterns Using Large Language Models](https://mlsanity.com/paper/2410.05488/) (2024)
- [A PRISMA-driven systematic mapping study on system assurance weakeners](https://mlsanity.com/paper/2311.08328/) (2023)
- [Arguing from Hazard Analysis in Safety Cases: A Modular Argument Pattern](https://mlsanity.com/paper/1704.03672/) (2017)
- [SmartGSN: an online tool to semi-automatically manage assurance cases](https://mlsanity.com/paper/2410.16675/) (2024)

## Abstract (arXiv)

Vision-Language-Action (VLA)-based driving systems represent a significant paradigm shift in autonomous driving since, by combining traffic scene understanding, linguistic interpretation, and action generation, these systems enable more flexible, adaptive, and instruction-responsive driving behaviors. However, despite their growing adoption and potential to support socially responsible autonomous driving as well as understanding high-level human instructions, VLA-based driving systems may exhibit new types of hazardous behaviors. For instance, the integration of open-ended natural language inputs (e.g., user or navigation instructions) into the multimodal control loop may lead to unpredictable and unsafe behaviors that could endanger vehicle occupants and pedestrians. Hence, assuring the safety of these systems is crucial to help build trust in their operations. To support this, we propose a novel safety case design approach called RAISE. Our approach introduces novel patterns tailored to instruction-based driving systems such as VLA-based driving systems, an extension of Hazard Analysis and Risk Assessment (HARA) detailing safe scenarios and their outcomes, and a design technique to create the safety cases of VLA-based driving systems. A case study on SimLingo illustrates how our approach can be used to construct rigorous, evidence-based safety claims for this emerging class of autonomous driving systems.