mlsanityData to 14 Oct 2026

OpenAI4S: Code as Action, Science as Sessions

arXiv 2609.15096 ↗First posted 14 Sep 2026cs.AI cs.CL cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

AI co-scientists could accelerate computational research, but over a long-running study the workflow also has to stay inspectable, resumable and reproducible, which requires persistent computational state and provenance. Here we present OpenAI4S, an open-source scientific research agent built around the principle of Code as Action, Science as Sessions. OpenAI4S combines a persistent computing runtime with research-session management: orchestration is handled through structured tool calls, while scientific actions are represented as complete code cells executed in persistent Python and R kernels. An append-only Action Ledger, per-cell execution records, versioned artifacts, environment records, and workspace checkpoints preserve how results were produced and support session recovery, branching, and extension. Configurable sandboxing, permission controls, and code and trajectory screening provide complementary safeguards. We evaluate OpenAI4S on 36 research scenarios spanning retrosynthesis, molecular dynamics, protein binder design, protein mutation, catalyst screening, and mineral spectroscopy, measuring scientific task accuracy, workflow completeness, and reproducibility of the resulting repositories. OpenAI4S achieves an overall score of 7.83, compared with 5.7--6.4 for a general-purpose coding harness evaluated with three frontier models, with the largest gains on long-horizon and computation-intensive workflows. These results suggest that integrating persistent execution with session-level provenance can improve the reliability of AI-assisted scientific workflows. Environment specification and full rerunnability remain weak for every evaluated system, ours included, so reproducibility is still an open problem for scientific agents. The system is available under the MIT license at hrefhttps://github.com/PKU-YuanGroup/OpenAI4Sgithub.com/PKU-YuanGroup/OpenAI4S.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 22 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

15 of 22 references.

References · 15

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  4. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  6. 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 366
  7. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
  8. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 178
  9. 2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 123
  10. 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
  11. 2023TaskWeaver: A Code-First Agent Frameworkcited by 86
  12. 2026EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discoverycited by 51
3 more
  1. 2025Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operationscited by 49
  2. 2026Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesiscited by 6
  3. 2026OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Deploy-Master: Automating the Deployment of 50,000+ Agent-Ready Scientific Tools in One Day
  2. 2026VERITAS: Towards a General-Purpose Replication Tool for Scientific Research
  3. 2026ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments
  4. 2026On Accelerating Grounded Code Development for Research
  5. 2025Re-opening open-source science through AI assisted development ↗
  6. 2026BloClaw: An Omniscient, Multi-Modal Agentic Workspace for Next-Generation Scientific Discovery
  7. 2026Research-Native by Construction: Minimal Nodes, Re-verifiable Workflows, and Compounding Memory for Long-Horizon Scientific Agents
  8. 2025AInsteinBench: Benchmarking Coding Agents on Scientific Repositories