mlsanityData to 14 Oct 2026

PaperCompiler: Faithful Paper-to-Code Generation via Repository-Level Specification Compilation

arXiv 2609.02272 ↗First posted 2 Sep 2026cs.CL cs.AI cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

Faithfully translating research papers into repository-level implementations remains challenging because papers often describe methods at a high level, leave implementation assumptions implicit, and require generated repositories to preserve method logic, evaluation protocols, and cross-file consistency. Despite recent advances in paper-to-code agents, their intermediate outputs are often presented as free-form plans or summaries that downstream coding agents may ignore, reinterpret, or compress, leading to algorithmic simplification and inconsistent repository structure. To address these challenges, we introduce PaperCompiler, a paper-to-code generation framework that compiles paper-grounded evidence into explicit repository-level implementation specifications. PaperCompiler grounds implementation-relevant evidence while preserving source provenance and distinguishing paper-supported, inferred, externally delegated, and unresolved information. The resulting specifications encode non-degradation requirements, ownership assignments, cross-file dependencies, and file-level constraints. Repository generation proceeds under these compiled specifications while retaining flexibility over local engineering choices not fixed by the paper. PaperCompiler outperforms strong baselines on Paper2CodeBench, achieving a 13.8% relative improvement in reference-based fidelity (from 3.64 to 4.15) and reducing high-severity evaluator critiques (from 13.2% to 6.1%).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

15 of 23 references.

References · 15

  1. 2021Program Synthesis with Large Language Modelscited by 3,637
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  5. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
  6. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
  7. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
  8. 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 183
  9. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 175
  10. 2023CodePlan: Repository-level Coding using LLMs and Planningcited by 140
  11. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 70
  12. 2023ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Codecited by 65
3 more
  1. 2023RepoFusion: Training Code Models to Understand Your Repositorycited by 40
  2. 2025AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineagecited by 32
  3. 2026HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Executioncited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DeepRepro: State-Aware Subplanning for Paper-to-Code Reproduction in Evolving Repositories
  2. 2026TICoder: A Repository-Level Code Generation Framework with Test-Driven Planning and Implementation-Aware Reuse ↗
  3. 2026SA-Bench: Evaluating Semantic Alignment in LLM-Based Paper Reproduction
  4. 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
  5. 2026ReCUBE: Evaluating Repository-Level Context Utilization in Code Generation
  6. 2026In Line with Context: Repository-Level Code Generation via Context Inlining
  7. 2026Remember Your Trace: Memory-Guided Long-Horizon Agentic Framework for Consistent and Hierarchical Repository-Level Code Documentation
  8. 2026Large-scale Repository Engineering via Agent-Native Reusable Code Primitives