mlsanityData to 7 Oct 2026

MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agents

arXiv 2408.14033 ↗First posted 26 Aug 2024cs.AI cs.CL cs.LGcited by 64

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Autonomous machine learning research has gained significant attention recently. We present MLR-COPILOT, an autonomous Machine Learning Research framework powered by large language model agents. The system is designed to enhance ML research productivity through automatic generation and implementation of research ideas within constraints. Our work was released in August 2024 (concurrent to AI-Scientist) and has gained notable recognition from leading projects. We further enhance our ideation with training afterwards. The framework consists of three stages: idea generation, experiment implementation, and code execution. First, existing research papers are used to generate feasible ideas and experiment plans with IdeaAgent, powered by an RL-tuned LLM. Next, ExperimentAgent leverages retrieved prototype code to convert plans into executable code with optionally retrieved candidate models and data from HuggingFace. In the final stage, ExperimentAgent runs experiments, and allows subsequent iterations of debugging and human feedback for a better chance of success with executable outcomes. We evaluate our framework on five machine learning research tasks. Experiment results demonstrate the potential of our framework to facilitate ML research progress and innovation.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 59 references that are in the map. First posted on arXiv 26 Aug 2024.

Builds on

14 of 59 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  5. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  6. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 174
  7. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 150
  8. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  9. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  10. 2023AutoML-GPT: Automatic Machine Learning with GPTcited by 80
  11. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 61
  12. 2023Large Language Models are Zero Shot Hypothesis Proposerscited by 45
2 more
  1. 2023Prompt2Model: Generating Deployable Models from Natural Language Instructionscited by 40
  2. 2017Scientific document summarization via citation contextualization and scientific discoursecited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 32; 2026: 20.

64 papers.

Sort:

By area:

  1. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  2. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
  3. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
  4. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  5. 2025Towards Scientific Intelligence: A Survey of LLM-based Scientific Agentscited by 96
  6. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 94
  7. 2025From Automation to Autonomy: A Survey on Large Language Models in Scientific Discoverycited by 86
  8. 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directionscited by 74
  9. 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypothesescited by 68
  10. 2024Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideascited by 52
  11. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  12. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
38 more
  1. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  2. 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 34
  3. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Researchcited by 29
  4. 2025MLRC-Bench: Can Language Agents Solve Machine Learning Research Challenges?cited by 27
  5. 2024AAAR-1.0: Assessing AI's Potential to Assist Researchcited by 24
  6. 2026AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaborationcited by 23
  7. 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 20
  8. 2025Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports From Scratch with Agentic Frameworkcited by 20
  9. 2025AutoMind: Adaptive Knowledgeable Agent for Automated Data Sciencecited by 20
  10. 2024Literature Meets Data: A Synergistic Approach to Hypothesis Generationcited by 19
  11. 2024AIGS: Generating Science from AI-Powered Automated Falsificationcited by 17
  12. 2025All That Glitters is Not Novel: Plagiarism in AI Generated Researchcited by 17
  13. 2025How Far Are AI Scientists from Changing the World?cited by 15
  14. 2025ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologiescited by 14
  15. 2025AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomycited by 13
  16. 2025CellForge: Agentic Design of Virtual Cell Modelscited by 11
  17. 2025MIR: Methodology Inspiration Retrieval for Scientific Research Problemscited by 10
  18. 2026Can We Predict Before Executing Machine Learning Agents?cited by 10
  19. 2025Matter-of-Fact: A Benchmark for Verifying the Feasibility of Literature-Supported Claims in Materials Sciencecited by 9
  20. 2025Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluatorcited by 9
  21. 2025AlphaResearch: Accelerating New Algorithm Discovery with Language Modelscited by 9
  22. 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 9
  23. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  24. 2025Automatic Evaluation Metrics for Artificially Generated Scientific Researchcited by 8
  25. 2025LLM-REVal: Can We Trust LLM Reviewers Yet?cited by 8
  26. 2024Creativity in AI: Progresses and Challengescited by 7
  27. 2025JoyAgent-JDGenie: Technical Report on the GAIAcited by 7
  28. 2025A Comprehensive Review of AI Agents: Transforming Possibilities in Technology and Beyondcited by 6
  29. 2025Evidence-Bound Autonomous Research (EviBound): A Governance Framework for Eliminating False Claimscited by 6
  30. 2026MLReplicate: Benchmarking Autonomous Research Systems for Machine Learning Reproducibilitycited by 6
  31. 2026FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamicscited by 6
  32. 2025The Curious Language Model: Strategic Test-Time Information Acquisitioncited by 5
  33. 2026FARS: A Fully Automated Research System Deployed at Scalecited by 5
  34. 2025A Human-AI Collaborative Workflow for Mathematical Discovery: A Case Study in Grover-Compatible Riemannian Optimizationcited by 4
  35. 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Modelscited by 3
  36. 2026ASI-Bench: At the Dawn of Artificial Superintelligencecited by 3
  37. 2024LDC: Learning to Generate Research Idea with Dynamic Controlcited by 2
  38. 2026AI-Supervisor: Autonomous AI Research Supervision via a Persistent Research World Modelcited by 2

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ML-AutoResearch: Training Machine Learning Research Agents with Automatically Generated Environments
  2. 2025MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research
  3. 2025From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking
  4. 2026DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories
  5. 2024Towards Automated Machine Learning Research
  6. 2025ML-Tool-Bench: Tool-Augmented Planning for ML Tasks
  7. 2026Machine Learning as a Tool (MLAT): A Framework for Integrating Statistical ML Models as Callable Tools within LLM Agent Workflows
  8. 2026Towards End-to-End Automation of AI Research