mlsanityData to 7 Oct 2026

N-Version Programming with Coding Agents

arXiv 2606.20158 ↗First posted 18 Jun 2026cs.SEcited by 2

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

This paper revisits the classical concept on N-version programming in the setting of contemporary AI coding agents. Revisiting the seminal Knight-Leveson experiment, we study whether diversity across agent systems, models, and implementation languages creates diverse failure modes. Using the Knight-Leveson's, Launch Interceptor Program Specification, we evaluate 48 agent-generated implementations on a shared oracle and a campaign of 1,000,000 randomized test inputs. The results show substantial common-mode failure, along the findings of Knight-Leveson. Further analysis that many of those co-occuring failures can be traced to where is specification is particularly hard or ambiguous. We also demonstrate that diversity from coding agents provides practical benefit: across majority voting three-version units, the mean failure count drops from 387.44 for single versions to 130.99 for triples, and 11,844 N-version units exhibit zero observed failures. Our original results is the strongest evidence to date that N-Version Programming with coding agents is a useful engineering strategy.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 8 of its 25 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

8 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Agentic Software Engineering: Foundational Pillars and a Research Roadmap7 Sep

Other references · 7

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  4. 2025Enhancing LLM Code Generation with Ensembles: A Similarity-Based Selection Approachcited by 10
  5. 2025Incoherence as Oracle-less Measure of Error in LLM-Based Code Generationcited by 6
  6. 2024Galapagos: Automated N-Version Programming with LLMscited by 5
  7. 2024N-Version Assessment and Enhancement of Generative AIcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Reliability Theory for AI Controlcited by 1
  2. 2026Builder, Defender, Breaker: Measurable Independence and Bounded Autonomy When Generative Models Build, Defend and Test Softwarecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures
  2. 2026EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions
  3. 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model
  4. 2026A Jagged Frontier: Evaluating Robustness of Code Agents to Semantics-Preserving Transformations
  5. 2026Understanding the Architecture of Coding Agents: An Exploratory Study Using a Research Prototype
  6. 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure
  7. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
  8. 2025Understanding and Bridging the Planner-Coder Gap: A Systematic Study on the Robustness of Multi-Agent Systems for Code Generation