Data to 7 Oct 2026

From Dead Code and Static Requirements to Working Engines: Software Revival with Coding Agents

arXiv 2609.36161 ↗First posted 28 Sep 2026cs.SE cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Can coding agents restore software that no longer runs while preserving its underlying methods, and reconstruct industrial software engines from open specifications? Here we introduce ReviveBench, a benchmark with two task families evaluated by hidden verifiers calibrated against native execution environments, established engineering tools, or purpose-built reference implementations. The revival family comprises ten tasks involving dependency incompatibilities, deleted core modules, legacy builds, and a GPU-based foundation model. Every starting workspace fails verification, and the strongest evaluated model passes all ten tasks in at least one run each. In contamination-control experiments, identifier obfuscation reduces line similarity to the original implementations from 0.51--0.96 to 0.03--0.44 without reducing the observed pass rate of any evaluated model. On repositories created after the stated knowledge cutoffs, the strongest model passes eight of nine runs. The reconstruction family comprises thirteen tasks spanning numerical, geometric, hardware, and transactional systems (e.g. CAD and CRM). Two models meet the benchmark's pass criteria on all thirteen, although our audit shows that the CFD task cannot establish numerical-solver capability. Benchmark construction and auditing uncover 28 verifier defects, including 24 false negatives and two false positives. These findings show that executable verification can itself introduce substantial measurement error. We present three practical checks: test whether prescribed methods can reach the grading thresholds, investigate agreement among independently generated candidates, and recompute diagnostics from submitted artifacts. ReviveBench thus provides both an evaluation of software revival and engine reconstruction, and cases in validating the verifiers used to measure coding agents for software design.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 36 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

35 of 36 references.

Closest ancestors · 6

Same direction, newest first

Other references · 29

2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
2021Program Synthesis with Large Language Modelscited by 3,509
2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
2024A Survey on LLM-as-a-Judgecited by 1,384
2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 246
2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 208
2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194

17 more

2023ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generationcited by 166
2023RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Modelcited by 143
2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 115
2024SWE-Bench+: Enhanced Coding Benchmark for LLMscited by 95
2024The Limits of Inference Scaling Through Resamplingcited by 51
2024Commit0: Library Generation from Scratchcited by 50
2025The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reasoncited by 47
2024MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmarkcited by 41
2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 25
2026LLMs Gaming Verifiers: RLVR can Lead to Reward Hackingcited by 24
2024LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systemscited by 20
2025An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliabilitycited by 17
2025RealBench: Benchmarking Verilog Generation Models with Real-World IP Designscited by 14
2026Benchmarking AI Agents for Addressing Scientific Challenges Across Scalescited by 10
2025Where Do LLMs Still Struggle? An In-Depth Analysis of Code Generation Benchmarkscited by 5
2026Before the Model Learns the Bug:Fuzzing RLVR Verifierscited by 5
2026Towards Understanding Specification Gaming in Reasoning Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way