mlsanityData to 7 Oct 2026

The Last Human-Written Paper: Agent-Native Research Artifacts

arXiv 2604.24658 ↗First posted 27 Apr 2026cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Scientific publication compresses a branching, iterative research process into a linear narrative, discarding the majority of what was discovered along the way. This compilation imposes two structural costs: a Storytelling Tax, where failed experiments, rejected hypotheses, and the branching exploration process are discarded to fit a linear narrative; and an Engineering Tax, where the gap between reviewer-sufficient prose and agent-sufficient specification leaves critical implementation details unwritten. Tolerable for human readers, these costs become critical when AI agents must understand, reproduce, and extend published work. We introduce the Agent-Native Research Artifact (ARA), a protocol that replaces the narrative paper with a machine-executable research package structured around four layers: scientific logic, executable code with full specifications, an exploration graph that preserves the failures compilation discards, and evidence grounding every claim in raw outputs. Three mechanisms support the ecosystem: a Live Research Manager that captures decisions and dead ends during ordinary development; an ARA Compiler that translates legacy PDFs and repos into ARAs; and an ARA-native review system that automates objective checks so human reviewers can focus on significance, novelty, and taste. On PaperBench and RE-Bench, ARA raises question-answering accuracy from 72.4% to 93.7% and reproduction success from 57.4% to 64.4%. On RE-Bench's five open-ended extension tasks, preserved failure traces in ARA accelerate progress, but can also constrain a capable agent from stepping outside the prior-run box depending on the agent's capabilities. Our code is open-sourced at https://github.com/Orchestra-Research/Agent-Native-Research-Artifact.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 40 of its 84 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

40 of 84 references.

References · 40

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  6. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  7. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  8. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  9. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  10. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
  11. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  12. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
28 more
  1. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  2. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
  3. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 208
  4. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 200
  5. 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 166
  6. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
  7. 2025Where LLM Agents Fail and How They can Learn From Failurescited by 118
  8. 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 105
  9. 2019An Open Source AutoML Benchmarkcited by 85
  10. 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
  11. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  12. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  13. 2025ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Codecited by 34
  14. 2026Scientific production in the era of Large Language Modelscited by 30
  15. 2021Packaging research artefacts with RO-Cratecited by 22
  16. 2025Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI Agentscited by 19
  17. 2026Autonomous Agents Coordinating Distributed Discovery Through Emergent Artifact Exchangecited by 11
  18. 2026SciCoQA: Quality Assurance for Scientific Paper--Code Alignmentcited by 9
  19. 2026From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agentscited by 7
  20. 2018Computing Environments for Reproducibility: Capturing the "Whole Tale"cited by 6
  21. 2026What Papers Don't Tell You: Recovering Tacit Knowledge for Automated Paper Reproductioncited by 6
  22. 2026Codified Context: Infrastructure for AI Agents in a Complex Codebasecited by 5
  23. 2025The Discovery Engine: A Framework for AI-Driven Synthesis and Navigation of Scientific Knowledge Landscapescited by 4
  24. 2024Rethinking the production and publication of machine-reusable expressions of research findingscited by 3
  25. 2025DecMetrics: Structured Claim Decomposition Scoring for Factually Consistent LLM Outputscited by 3
  26. 2026Sci-Reasoning: A Dataset Decoding AI Innovation Patternscited by 2
  27. 2026Operationalising Artificial Intelligence Bills of Materials (AIBOMs) for Verifiable AI Provenance and Lifecycle Assurancecited by 2
  28. 2025What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representationscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.