mlsanityData to 14 Oct 2026

Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents

arXiv 2502.16069 ↗First posted 22 Feb 2025cs.AI cs.LGcited by 33

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Scientific experimentation, a cornerstone of human progress, demands rigor in reliability, methodical control, and interpretability to yield meaningful results. Despite the growing capabilities of large language models (LLMs) in automating different aspects of the scientific process, automating rigorous experimentation remains a significant challenge. To address this gap, we propose Curie, an AI agent framework designed to embed rigor into the experimentation process through three key components: an intra-agent rigor module to enhance reliability, an inter-agent rigor module to maintain methodical control, and an experiment knowledge module to enhance interpretability. To evaluate Curie, we design a novel experimental benchmark composed of 46 questions across four computer science domains, derived from influential research papers, and widely adopted open-source projects. Compared to the strongest baseline tested, we achieve a 3.4× improvement in correctly answering experimental questions. Curie is open-sourced at https://github.com/Just-Curieous/Curie.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 30 of its 41 references that are in the map. First posted on arXiv 22 Feb 2025.

Builds on

30 of 41 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Agent Laboratory: Using LLM Agents as Research Assistants8 Jan
  2. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery7 Oct
  3. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning9 Sep
  4. 2024Data Interpreter: An LLM Agent For Data Science28 Feb
  5. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents19 Apr

Other references · 25

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  7. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  8. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  9. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  10. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  11. 2023Large Language Modelscited by 780
  12. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
13 more
  1. 2023Mathematical Capabilities of ChatGPTcited by 268
  2. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
  3. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 153
  4. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 120
  5. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 115
  6. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  7. 2024Hypothesis Generation with Large Language Modelscited by 71
  8. 2023MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Taskscited by 66
  9. 2024LitLLM: A Toolkit for Scientific Literature Reviewcited by 63
  10. 2024AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic Reviewscited by 44
  11. 2023Accelerating science with human-aware artificial intelligencecited by 28
  12. 2023The HaLLMark Effect: Supporting Provenance and Transparent Use of Large Language Models in Writing with Interactive Visualizationcited by 24
  13. 2023Automated Scientific Discovery: From Equation Discovery to Autonomous Discovery Systemscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 18.

33 papers.

Sort:

By area:

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  2. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 93
  3. 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 72
  4. 2025LLM Agents for Education: Advances and Applicationscited by 59
  5. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  6. 2025Measuring Agents in Productioncited by 37
  7. 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 21
  8. 2025A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Taskscited by 19
  9. 2026ScientistOne: Towards Human-Level Autonomous Research via Chain-of-Evidencecited by 19
  10. 2025RExBench: Can coding agents autonomously implement AI research extensions?cited by 18
  11. 2025CellForge: Agentic Design of Virtual Cell Modelscited by 13
  12. 2025A Survey of AI Scientistscited by 11
21 more
  1. 2026AI for Auto-Research: Roadmap & User Guidecited by 11
  2. 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 10
  3. 2026Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistancecited by 9
  4. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  5. 2026MLReplicate: Benchmarking Autonomous Research Systems for Machine Learning Reproducibilitycited by 8
  6. 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoningcited by 4
  7. 2025Idea2Plan: Exploring AI-Powered Research Planningcited by 4
  8. 2026Sci-Reasoning: A Dataset Decoding AI Innovation Patternscited by 4
  9. 2026ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environmentscited by 4
  10. 2025OpenLens AI: Fully Autonomous Research Agent for Health Infomaticscited by 3
  11. 2026Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraudcited by 2
  12. 2025Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecturecited by 1
  13. 2026Experiment-as-Code Labs: A Declarative Stack for AI-Driven Scientific Discoverycited by 1
  14. 2026Terminal Agents: A Survey of AI Agents in Command-Line Environmentscited by 1
  15. 2025PAME-AI: Patient Messaging Creation and Optimization using Agentic AIcited by 0
  16. 2026Hypothesis-Driven Deep Research with Large Language Models: A Structured Methodology for Automated Knowledge Discoverycited by 0
  17. 2026Simulate, Reason, Decide: Scientific Reasoning with LLMs for Simulation-Driven Decision Makingcited by 0
  18. 2026Agentic AutoResearch forSpace Autonomy: An Auditable, LLM-Driven Research Agent for Aerospace Control Problemscited by 0
  19. 2026AgentGUI: An Interface for Observing and Steering Long-Running AI Agentscited by 0
  20. 2026CrossAudit: A Git-Native, Cross-Vendor Audit Loop for Agentic Sciencecited by 0
  21. 2026SHELDON: Reasoning through Dual Search over Hypothesis and Observation Spacescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentIdeaBench: Benchmarking Scientific Ideation in the Agent Era
  2. 2026AI scientists produce results without reasoning scientifically
  3. 2025From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking
  4. 2025InnovatorBench: Evaluating Agents' Ability to Conduct Innovative LLM Research
  5. 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
  6. 2026Towards End-to-End Automation of AI Research
  7. 2026AIBuildAI-2: A Knowledge-Enhanced Agent for Automatically Building AI Models
  8. 2025AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite