# N-Version Assessment and Enhancement of Generative AI

arXiv: https://arxiv.org/abs/2409.14071 · HTML: https://mlsanity.com/paper/2409.14071/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 15 references that are in the map.
- First posted on arXiv 21 Sep 2024.

## Builds on (most cited first, 4 of 4)

- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Large Language Models and Simple, Stupid Bugs](https://mlsanity.com/paper/2303.11455/) (2023)
- [The Multiple Facets of Software Diversity: Recent Developments in Year 2000 and Beyond](https://mlsanity.com/paper/1409.7324/) (2014)
- [Morescient GAI for Software Engineering (Extended Version)](https://mlsanity.com/paper/2406.04710/) (2024)

## Built on by (most cited first, 2 of 2)

- [N-Version Programming with Coding Agents](https://mlsanity.com/paper/2606.20158/) (2026)
- [Test-driven Software Experimentation with LASSO: an LLM Prompt Benchmarking Example](https://mlsanity.com/paper/2410.08911/) (2024)

## Abstract (arXiv)

Generative AI (GAI) holds great potential to improve software engineering productivity, but its untrustworthy outputs, particularly in code synthesis, pose significant challenges. The need for extensive verification and validation (V&V) of GAI-generated artifacts may undermine the potential productivity gains. This paper proposes a way of mitigating these risks by exploiting GAI's ability to generate multiple versions of code and tests to facilitate comparative analysis across versions. Rather than relying on the quality of a single test or code module, this "differential GAI" (D-GAI) approach promotes more reliable quality evaluation through version diversity. We introduce the Large-Scale Software Observatorium (LASSO), a platform that supports D-GAI by executing and analyzing large sets of code versions and tests. We discuss how LASSO enables rigorous evaluation of GAI-generated artifacts and propose its application in both software development and GAI research.