# Testing learning-enabled cyber-physical systems with Large-Language Models: A Formal Approach

arXiv: https://arxiv.org/abs/2311.07377 · HTML: https://mlsanity.com/paper/2311.07377/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 80 references that are in the map.
- First posted on arXiv 13 Nov 2023.

## Builds on (most cited first, 27 of 27)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks](https://mlsanity.com/paper/1702.01135/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- [On a Formal Model of Safe and Scalable Self-driving Cars](https://mlsanity.com/paper/1708.06374/) (2017)
- [Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models](https://mlsanity.com/paper/2212.14834/) (2022)
- [Scenic: A Language for Scenario Specification and Scene Generation](https://mlsanity.com/paper/1809.09310/) (2018)
- [Verification of Deep Convolutional Neural Networks Using ImageStars](https://mlsanity.com/paper/2004.05511/) (2020)
- [The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Results](https://mlsanity.com/paper/2109.00498/) (2021)
- [An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models](https://mlsanity.com/paper/2002.02175/) (2020)
- [Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systems](https://mlsanity.com/paper/2001.10494/) (2020)
- [Learning Linear Temporal Properties](https://mlsanity.com/paper/1806.03953/) (2018)
- [Scenic: A Language for Scenario Specification and Data Generation](https://mlsanity.com/paper/2010.06580/) (2020)
- [ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Model](https://mlsanity.com/paper/2304.03086/) (2023)
- [How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generation](https://mlsanity.com/paper/2307.12469/) (2023)
- [Testing of Autonomous Driving Systems: Where Are We and Where Should We Go?](https://mlsanity.com/paper/2106.12233/) (2021)
- [Neural Network Guided Evolutionary Fuzzing for Finding Traffic Violations of Autonomous Vehicles](https://mlsanity.com/paper/2109.06126/) (2021)
- [Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems](https://mlsanity.com/paper/2210.15432/) (2022)
- [MMFN: Multi-Modal-Fusion-Net for End-to-End Driving](https://mlsanity.com/paper/2207.00186/) (2022)
- [LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehicles](https://mlsanity.com/paper/2208.14656/) (2022)
- [A Declarative Metamorphic Testing Framework for Autonomous Driving](https://mlsanity.com/paper/2012.10672/) (2020)
- [Scenario-Based Test Reduction and Prioritization for Multi-Module Autonomous Driving Systems](https://mlsanity.com/paper/2209.01546/) (2022)
- [Linear-time Temporal Logic guided Greybox Fuzzing](https://mlsanity.com/paper/2109.02312/) (2021)
- [GARL: Genetic Algorithm-Augmented Reinforcement Learning to Detect Violations in Marker-Based Autonomous Landing Systems](https://mlsanity.com/paper/2310.07378/) (2023)
- [Fully Automated Verification of Linear Time-Invariant Systems against Signal Temporal Logic Specifications via Reachability Analysis](https://mlsanity.com/paper/2306.04089/) (2023)

## Built on by (most cited first, 5 of 5)

- [NeuroStrata: Harnessing Neurosymbolic Paradigms for Improved Design, Testability, and Verifiability of Autonomous CPS](https://mlsanity.com/paper/2502.12267/) (2025)
- [3D Software Synthesis Driven by Constraint-Expressive Intermediate Representation](https://mlsanity.com/paper/2507.18625/) (2025)
- [Lumos: Let there be Language Model System Certification](https://mlsanity.com/paper/2512.02966/) (2025)
- [SafePilot: A Framework for Assuring LLM-enabled Cyber-Physical Systems](https://mlsanity.com/paper/2603.21523/) (2026)
- [Agentic Generation and Evolution of Knowledge Models](https://mlsanity.com/paper/2606.03662/) (2026)

## Abstract (arXiv)

The integration of machine learning (ML) into cyber-physical systems (CPS) offers significant benefits, including enhanced efficiency, predictive capabilities, real-time responsiveness, and the enabling of autonomous operations. This convergence has accelerated the development and deployment of a range of real-world applications, such as autonomous vehicles, delivery drones, service robots, and telemedicine procedures. However, the software development life cycle (SDLC) for AI-infused CPS diverges significantly from traditional approaches, featuring data and learning as two critical components. Existing verification and validation techniques are often inadequate for these new paradigms. In this study, we pinpoint the main challenges in ensuring formal safety for learningenabled CPS.We begin by examining testing as the most pragmatic method for verification and validation, summarizing the current state-of-the-art methodologies. Recognizing the limitations in current testing approaches to provide formal safety guarantees, we propose a roadmap to transition from foundational probabilistic testing to a more rigorous approach capable of delivering formal assurance.