# N-Version Programming with Coding Agents

arXiv: https://arxiv.org/abs/2606.20158 · HTML: https://mlsanity.com/paper/2606.20158/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 8 of 8)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [Enhancing LLM Code Generation with Ensembles: A Similarity-Based Selection Approach](https://mlsanity.com/paper/2503.15838/) (2025)
- [Incoherence as Oracle-less Measure of Error in LLM-Based Code Generation](https://mlsanity.com/paper/2507.00057/) (2025)
- [Galapagos: Automated N-Version Programming with LLMs](https://mlsanity.com/paper/2408.09536/) (2024)
- [N-Version Assessment and Enhancement of Generative AI](https://mlsanity.com/paper/2409.14071/) (2024)

## Built on by (most cited first, 2 of 2)

- [Reliability Theory for AI Control](https://mlsanity.com/paper/2609.26419/) (2026)
- [Builder, Defender, Breaker: Measurable Independence and Bounded Autonomy When Generative Models Build, Defend and Test Software](https://mlsanity.com/paper/2607.03215/) (2026)

## Abstract (arXiv)

This paper revisits the classical concept on N-version programming in the setting of contemporary AI coding agents. Revisiting the seminal Knight-Leveson experiment, we study whether diversity across agent systems, models, and implementation languages creates diverse failure modes. Using the Knight-Leveson's, Launch Interceptor Program Specification, we evaluate 48 agent-generated implementations on a shared oracle and a campaign of 1,000,000 randomized test inputs. The results show substantial common-mode failure, along the findings of Knight-Leveson. Further analysis that many of those co-occuring failures can be traced to where is specification is particularly hard or ambiguous. We also demonstrate that diversity from coding agents provides practical benefit: across majority voting three-version units, the mean failure count drops from 387.44 for single versions to 130.99 for triples, and 11,844 N-version units exhibit zero observed failures. Our original results is the strongest evidence to date that N-Version Programming with coding agents is a useful engineering strategy.