# On Integrating Large Language Models and Scenario-Based Programming for Improving Software Reliability

arXiv: https://arxiv.org/abs/2509.09194 · HTML: https://mlsanity.com/paper/2509.09194/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 55 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Clover: Closed-Loop Verifiable Code Generation](https://mlsanity.com/paper/2310.17807/) (2023)
- [Capabilities of GPT-5 on Multimodal Medical Reasoning](https://mlsanity.com/paper/2508.08224/) (2025)
- [AI Governance and Accountability: An Analysis of Anthropic's Claude](https://mlsanity.com/paper/2407.01557/) (2024)
- [LLMs: A Game-Changer for Software Engineers?](https://mlsanity.com/paper/2411.00932/) (2024)
- [Guiding Enumerative Program Synthesis with Large Language Models](https://mlsanity.com/paper/2403.03997/) (2024)
- [Guarded Deep Learning using Scenario-Based Modeling](https://mlsanity.com/paper/2006.03863/) (2020)
- [On-the-Fly Construction of Composite Events in Scenario-Based Modeling using Constraint Solvers](https://mlsanity.com/paper/1909.00408/) (2019)
- [Towards Repairing Scenario-Based Models with Rich Events](https://mlsanity.com/paper/2101.03504/) (2021)
- [Enhancing Deep Learning with Scenario-Based Override Rules: a Case Study](https://mlsanity.com/paper/2301.08114/) (2023)
- [On Augmenting Scenario-Based Modeling with Generative AI](https://mlsanity.com/paper/2401.02245/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) are fast becoming indispensable tools for software developers, assisting or even partnering with them in crafting complex programs. The advantages are evident -- LLMs can significantly reduce development time, generate well-organized and comprehensible code, and occasionally suggest innovative ideas that developers might not conceive on their own. However, despite their strengths, LLMs will often introduce significant errors and present incorrect code with persuasive confidence, potentially misleading developers into accepting flawed solutions. In order to bring LLMs into the software development cycle in a more reliable manner, we propose a methodology for combining them with ``traditional'' software engineering techniques in a structured way, with the goal of streamlining the development process, reducing errors, and enabling users to verify crucial program properties with increased confidence. Specifically, we focus on the Scenario-Based Programming (SBP) paradigm -- an event-driven, scenario-based approach for software engineering -- to allow human developers to pour their expert knowledge into the LLM, as well as to inspect and verify its outputs. To evaluate our methodology, we conducted a significant case study, and used it to design and implement the Connect4 game. By combining LLMs and SBP we were able to create a highly-capable agent, which could defeat various strong existing agents. Further, in some cases, we were able to formally verify the correctness of our agent. Finally, our experience reveals interesting insights regarding the ease-of-use of our proposed approach. The full code of our case-study will be made publicly available with the final version of this paper.