# Engineering Safety Requirements for Autonomous Driving with Large Language Models

arXiv: https://arxiv.org/abs/2403.16289 · HTML: https://mlsanity.com/paper/2403.16289/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 24 Mar 2024.

## Placed in

- End-to-end autonomous driving / [Large language models for driving](https://mlsanity.com/driving/large-language-models-for-driving/)

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [6-Layer Model for a Structured Description and Categorization of Urban Traffic and Environment](https://mlsanity.com/paper/2012.06319/) (2020)
- [Safety Analysis in the Era of Large Language Models: A Case Study of STPA using ChatGPT](https://mlsanity.com/paper/2304.01246/) (2023)
- [ChatGPT is on the Horizon: Could a Large Language Model be Suitable for Intelligent Traffic Safety Research and Applications?](https://mlsanity.com/paper/2303.05382/) (2023)
- [Can Large Language Models assist in Hazard Analysis?](https://mlsanity.com/paper/2303.15473/) (2023)
- [Welcome Your New AI Teammate: On Safety Analysis by Leashing Large Language Models](https://mlsanity.com/paper/2403.09565/) (2024)
- [Ethical Interviews in Software Engineering](https://mlsanity.com/paper/1906.07993/) (2019)
- [An Industrial Experience Report about Challenges from Continuous Monitoring, Improvement, and Deployment for Autonomous Driving Features](https://mlsanity.com/paper/2403.09474/) (2024)
- [On STPA for Distributed Development of Safe Autonomous Driving: An Interview Study](https://mlsanity.com/paper/2403.09509/) (2024)

## Built on by (most cited first, 14 of 14)

- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [Generating and Evolving Reward Functions for Highway Driving with Large Language Models](https://mlsanity.com/paper/2406.10540/) (2024)
- [XLM for Autonomous Driving Systems: A Comprehensive Review](https://mlsanity.com/paper/2409.10484/) (2024)
- [On Simulation-Guided LLM-based Code Generation for Safe Autonomous Driving Software](https://mlsanity.com/paper/2504.02141/) (2025)
- [Can LLMs Generate User Stories and Assess Their Quality?](https://mlsanity.com/paper/2507.15157/) (2025)
- [Large Language Models in Code Co-generation for Safe Autonomous Vehicles](https://mlsanity.com/paper/2505.19658/) (2025)
- [A Systematic Literature Review on Safety of the Intended Functionality for Automated Driving Systems](https://mlsanity.com/paper/2503.02498/) (2025)
- [Working Document -- Formalising Software Requirements with Large Language Models](https://mlsanity.com/paper/2506.14627/) (2025)
- [Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices](https://mlsanity.com/paper/2506.15028/) (2025)
- [Do AI Models Dream of Faster Code? An Empirical Study on LLM-Proposed Performance Improvements in Real-World Software](https://mlsanity.com/paper/2510.15494/) (2025)
- [Traceable LLM-Generated Hazard Scenarios for Operational Safety Analysis of Aviation Systems Using ASRS Reports](https://mlsanity.com/paper/2608.04697/) (2026)
- [From Safety Documentation to Safety Knowledge Support: An Evidence-Grounded LLM Framework for Medical Devices](https://mlsanity.com/paper/2608.12025/) (2026)
- [Context operations to architecture modelling output from large language models and evaluation criteria for their use in systems engineering design](https://mlsanity.com/paper/2609.10132/) (2026)

## Abstract (arXiv)

Changes and updates in the requirement artifacts, which can be frequent in the automotive domain, are a challenge for SafetyOps. Large Language Models (LLMs), with their impressive natural language understanding and generating capabilities, can play a key role in automatically refining and decomposing requirements after each update. In this study, we propose a prototype of a pipeline of prompts and LLMs that receives an item definition and outputs solutions in the form of safety requirements. This pipeline also performs a review of the requirement dataset and identifies redundant or contradictory requirements. We first identified the necessary characteristics for performing HARA and then defined tests to assess an LLM's capability in meeting these criteria. We used design science with multiple iterations and let experts from different companies evaluate each cycle quantitatively and qualitatively. Finally, the prototype was implemented at a case company and the responsible team evaluated its efficiency.