The Last Human-Written Paper: Agent-Native Research Artifacts
Not on the map. Why?
Abstract (arXiv)
Scientific publication compresses a branching, iterative research process into a linear narrative, discarding the majority of what was discovered along the way. This compilation imposes two structural costs: a Storytelling Tax, where failed experiments, rejected hypotheses, and the branching exploration process are discarded to fit a linear narrative; and an Engineering Tax, where the gap between reviewer-sufficient prose and agent-sufficient specification leaves critical implementation details unwritten. Tolerable for human readers, these costs become critical when AI agents must understand, reproduce, and extend published work. We introduce the Agent-Native Research Artifact (ARA), a protocol that replaces the narrative paper with a machine-executable research package structured around four layers: scientific logic, executable code with full specifications, an exploration graph that preserves the failures compilation discards, and evidence grounding every claim in raw outputs. Three mechanisms support the ecosystem: a Live Research Manager that captures decisions and dead ends during ordinary development; an ARA Compiler that translates legacy PDFs and repos into ARAs; and an ARA-native review system that automates objective checks so human reviewers can focus on significance, novelty, and taste. On PaperBench and RE-Bench, ARA raises question-answering accuracy from 72.4% to 93.7% and reproduction success from 57.4% to 64.4%. On RE-Bench's five open-ended extension tasks, preserved failure traces in ARA accelerate progress, but can also constrain a capable agent from stepping outside the prior-run box depending on the agent's capabilities. Our code is open-sourced at https://github.com/Orchestra-Research/Agent-Native-Research-Artifact.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 40 of its 84 references that are in the map. First posted on arXiv 27 Apr 2026.
Builds on
40 of 84 references.
References · 40
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
- 2020Fact or Fiction: Verifying Scientific Claimscited by 546
- 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
- 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
- 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
28 more
- 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
- 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 208
- 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 200
- 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 166
- 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
- 2025Where LLM Agents Fail and How They can Learn From Failurescited by 118
- 2025AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?cited by 105
- 2019An Open Source AutoML Benchmarkcited by 85
- 2021HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenMLcited by 75
- 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
- 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
- 2025ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Codecited by 34
- 2026Scientific production in the era of Large Language Modelscited by 30
- 2021Packaging research artefacts with RO-Cratecited by 22
- 2025Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI Agentscited by 19
- 2026Autonomous Agents Coordinating Distributed Discovery Through Emergent Artifact Exchangecited by 11
- 2026SciCoQA: Quality Assurance for Scientific Paper--Code Alignmentcited by 9
- 2026From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agentscited by 7
- 2018Computing Environments for Reproducibility: Capturing the "Whole Tale"cited by 6
- 2026What Papers Don't Tell You: Recovering Tacit Knowledge for Automated Paper Reproductioncited by 6
- 2026Codified Context: Infrastructure for AI Agents in a Complex Codebasecited by 5
- 2025The Discovery Engine: A Framework for AI-Driven Synthesis and Navigation of Scientific Knowledge Landscapescited by 4
- 2024Rethinking the production and publication of machine-reusable expressions of research findingscited by 3
- 2025DecMetrics: Structured Claim Decomposition Scoring for Factually Consistent LLM Outputscited by 3
- 2026Sci-Reasoning: A Dataset Decoding AI Innovation Patternscited by 2
- 2026Operationalising Artificial Intelligence Bills of Materials (AIBOMs) for Verifiable AI Provenance and Lifecycle Assurancecited by 2
- 2025What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representationscited by 1
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 25
- 2026Closing the Loop to Discover Psychological Theories with an Automated Cognitive Scientistcited by 6
- 2026Artifact-centered Claim-aware Observability for Autonomous Scientific Agentscited by 2
- 2026The Greatness of Science Cannot Be Planned: Agentic Auto-Research is Fuzz Testingcited by 1
- 2026Toward an Engineering of Science: Rebalancing Generation and Verification in the Age of AIcited by 0
- 2026Clarus: Coordinating Autonomous Research Agents toward Web-Scale Scientific Collaborationcited by 0
- 2026What is Left for Us? Second Scholarship Against the Degradation of Research by AIcited by 0
- 2026Position: AI Agents in Scientific Teams Should Be Studied as Human-Agent Systemscited by 0
- 2026PhenoBench: Mapping What a Deeply Phenotyped Human Cohort Can Tell Uscited by 0
- 2026Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognitioncited by 0
- 2026Making AI-Assisted Claims Independently Challengeable: Publication Authority and a Protocol for Falsifiable Publication Recordscited by 0
- 2026Gricea: An Open Science Platform for Conversational AI Researchcited by 0
- 2026Gricea: An Open Science Platform for Conversational AI Research18 Sep 2026
- 2026Making AI-Assisted Claims Independently Challengeable: Publication Authority and a Protocol for Falsifiable Publication Records15 Sep 2026
- 2026Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognition7 Sep 2026
- 2026PhenoBench: Mapping What a Deeply Phenotyped Human Cohort Can Tell Us5 Sep 2026
- 2026Artifact-centered Claim-aware Observability for Autonomous Scientific Agents18 Aug 2026
- 2026The Greatness of Science Cannot Be Planned: Agentic Auto-Research is Fuzz Testing10 Aug 2026
- 2026Position: AI Agents in Scientific Teams Should Be Studied as Human-Agent Systems2 Aug 2026
- 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops8 Jul 2026
- 2026What is Left for Us? Second Scholarship Against the Degradation of Research by AI4 Jul 2026
- 2026Clarus: Coordinating Autonomous Research Agents toward Web-Scale Scientific Collaboration29 Jun 2026
- 2026Closing the Loop to Discover Psychological Theories with an Automated Cognitive Scientist24 Jun 2026
- 2026Toward an Engineering of Science: Rebalancing Generation and Verification in the Age of AI11 May 2026