Can LLMs Enable Verification in Mainstream Programming?
Not on the map. Why?
Abstract (arXiv)
Although formal methods are capable of producing reliable software, they have seen minimal adoption in everyday programming. Automatic code generation using large language models is becoming increasingly widespread, but it rarely considers producing strong correctness guarantees. In this study, we explore the ability of LLMs to produce verified code in three verification languages (Dafny, Nagini, and Verus). To do so, we use manually curated datasets derived from the state-ofthe-art Python benchmark, HumanEval. We also assess what types of information are sufficient to achieve good-quality results.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 15 of its 33 references that are in the map. First posted on arXiv 18 Mar 2025.
Builds on
15 of 33 references.
References · 15
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Clover: Closed-Loop Verifiable Code Generationcited by 67
- 2024Towards AI-Assisted Synthesis of Verified Dafny Methodscited by 58
- 2023Can Large Language Models Transform Natural Language Intent into Formal Method Postconditions?cited by 57
- 2023Verus: Verifying Rust Programs using Linear Ghost Types (extended version)cited by 53
- 2023Finding Inductive Loop Invariants using Large Language Modelscited by 36
- 2024AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinementcited by 35
- 2024Automated Proof Generation for Rust Code via Self-Evolutioncited by 33
- 2024Towards Neural Synthesis for SMT-Assisted Proof-Oriented Programmingcited by 20
- 2023Leveraging Large Language Models for Automated Proof Synthesis in Rustcited by 11
- 2024dafny-annotator: AI-Assisted Verification of Dafny Programscited by 8
- 2019SLING: Using Dynamic Analysis to Infer Program Invariants in Separation Logiccited by 7
Built on by
Citing papers per year: 2026: 3.
3 papers.