PaperCompiler: Faithful Paper-to-Code Generation via Repository-Level Specification Compilation
Not on the map. Why?
Abstract (arXiv)
Faithfully translating research papers into repository-level implementations remains challenging because papers often describe methods at a high level, leave implementation assumptions implicit, and require generated repositories to preserve method logic, evaluation protocols, and cross-file consistency. Despite recent advances in paper-to-code agents, their intermediate outputs are often presented as free-form plans or summaries that downstream coding agents may ignore, reinterpret, or compress, leading to algorithmic simplification and inconsistent repository structure. To address these challenges, we introduce PaperCompiler, a paper-to-code generation framework that compiles paper-grounded evidence into explicit repository-level implementation specifications. PaperCompiler grounds implementation-relevant evidence while preserving source provenance and distinguishing paper-supported, inferred, externally delegated, and unresolved information. The resulting specifications encode non-degradation requirements, ownership assignments, cross-file dependencies, and file-level constraints. Repository generation proceeds under these compiled specifications while retaining flexibility over local engineering choices not fixed by the paper. PaperCompiler outperforms strong baselines on Paper2CodeBench, achieving a 13.8% relative improvement in reference-based fidelity (from 3.64 to 4.15) and reducing high-severity evaluator critiques (from 13.2% to 6.1%).
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
15 of 23 references.
References · 15
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
- 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
- 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 183
- 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 175
- 2023CodePlan: Repository-level Coding using LLMs and Planningcited by 140
- 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 70
- 2023ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Codecited by 65
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DeepRepro: State-Aware Subplanning for Paper-to-Code Reproduction in Evolving Repositories
- 2026TICoder: A Repository-Level Code Generation Framework with Test-Driven Planning and Implementation-Aware Reuse ↗
- 2026SA-Bench: Evaluating Semantic Alignment in LLM-Based Paper Reproduction
- 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
- 2026ReCUBE: Evaluating Repository-Level Context Utilization in Code Generation
- 2026In Line with Context: Repository-Level Code Generation via Context Inlining
- 2026Remember Your Trace: Memory-Guided Long-Horizon Agentic Framework for Consistent and Hierarchical Repository-Level Code Documentation
- 2026Large-scale Repository Engineering via Agent-Native Reusable Code Primitives