mlsanityData to 7 Oct 2026

ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models

arXiv 2404.07738 ↗First posted 11 Apr 2024cs.CL cs.AI cs.LGcited by 200

Not on the map. Why?

Cited by 88 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The pace of scientific research, vital for improving human life, is complex, slow, and needs specialized expertise. Meanwhile, novel, impactful research often stems from both a deep understanding of prior work, and a cross-pollination of ideas across domains and fields. To enhance the productivity of researchers, we propose ResearchAgent, which leverages the encyclopedic knowledge and linguistic reasoning capabilities of Large Language Models (LLMs) to assist them in their work. This system automatically defines novel problems, proposes methods and designs experiments, while iteratively refining them based on the feedback from collaborative LLM-powered reviewing agents. Specifically, starting with a core scientific paper, ResearchAgent is augmented not only with relevant publications by connecting information over an academic graph but also entities retrieved from a knowledge store derived from shared underlying concepts mined across numerous papers. Then, mimicking a scientific approach to improving ideas with peer discussions, we leverage multiple LLM-based ReviewingAgents that provide reviews and feedback via iterative revision processes. These reviewing agents are instantiated with human preference-aligned LLMs whose criteria for evaluation are elicited from actual human judgments via LLM prompting. We experimentally validate our ResearchAgent on scientific publications across multiple disciplines, showing its effectiveness in generating novel, clear, and valid ideas based on both human and model-based evaluation results. Our initial foray into AI-mediated scientific research has important implications for the development of future systems aimed at supporting researchers in their ideation and operationalization of novel work.

Key facts. Cited by 200 papers in the map (data through 7 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

22 of 41 references.

References · 22

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  5. 2023Qwen Technical Reportcited by 3,273
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  8. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  9. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  10. 2022Generating Sequences by Learning to Self-Correctcited by 271
  11. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  12. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
10 more
  1. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  2. 2023Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answeringcited by 64
  3. 2023Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestioncited by 43
  4. 2023CoQuest: Exploring Research Question Co-Creation with an LLM-based Agentcited by 34
  5. 2023The ART of LLM Refinement: Ask, Refine, and Trustcited by 29
  6. 2021Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Studycited by 23
  7. 2018Over-Optimization of Academic Publishing Metrics: Observing Goodhart's Law in Actioncited by 22
  8. 2022A Computational Inflection for Scientific Discoverycited by 22
  9. 2020AGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approachcited by 21
  10. 2023We are Who We Cite: Bridges of Influence Between Natural Language Processing and Other Academic Fieldscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 84; 2026: 94.

200 papers.

Sort:

By area:

  1. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  2. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  3. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  4. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  5. 2026Towards End-to-End Automation of AI Researchcited by 150
  6. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  7. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  8. 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 111
  9. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 94
  10. 2025AI-Researcher: Autonomous Scientific Innovationcited by 88
  11. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 84
  12. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
38 more
  1. 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directionscited by 74
  2. 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discoverycited by 74
  3. 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypothesescited by 68
  4. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  5. 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 64
  6. 2025MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Researchcited by 57
  7. 2025DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressivelycited by 56
  8. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
  9. 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injectionscited by 44
  10. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 41
  11. 2025Automated Hypothesis Validation with Agentic Sequential Falsificationscited by 40
  12. 2026Code as Agent Harnesscited by 40
  13. 2024FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Modelscited by 39
  14. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 36
  15. 2024ResearchTown: Simulator of Human Research Communitycited by 35
  16. 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 34
  17. 2025AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineagecited by 29
  18. 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Researchcited by 29
  19. 2025DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulationcited by 29
  20. 2025AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprisecited by 28
  21. 2025MLRC-Bench: Can Language Agents Solve Machine Learning Research Challenges?cited by 27
  22. 2025UserRL: Training Interactive User-Centric Agent via Reinforcement Learningcited by 27
  23. 2024Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Contextcited by 25
  24. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 25
  25. 2024SciPIP: An LLM-based Scientific Paper Idea Proposercited by 22
  26. 2025OptimAI: Optimization from Natural Language Using LLM-Powered AI Agentscited by 22
  27. 2024Generating Interesting Scientific Ideas using Knowledge Graphs and LLMs: Evaluations with 100 Research Group Leaderscited by 21
  28. 2024AIGS: Generating Science from AI-Powered Automated Falsificationcited by 17
  29. 2025GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluationcited by 17
  30. 2025Automatic Red Teaming LLM-based Agents with Model Context Protocol Toolscited by 17
  31. 2026AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discoverycited by 17
  32. 2024LLMs can Realize Combinatorial Creativity: Generating Creative Ideas via LLMs for Scientific Researchcited by 16
  33. 2025OmniScientist: Toward a Co-evolving Ecosystem of Human and AI Scientistscited by 15
  34. 2025ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologiescited by 14
  35. 2025Gravity-Bench-v1: A Benchmark on Gravitational Physics Discovery for Agentscited by 13
  36. 2025PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priorscited by 13
  37. 2025Reinforcement Learning for Machine Learning Engineering Agentscited by 13
  38. 2026FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insightscited by 13

50 most cited of 200 All 200 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025AISSISTANT: Human-AI Collaborative Review and Perspective Research Workflows in Data Science
  2. 2026SurveyAgent-HKA: A multi-agent framework for scientific survey generation with LLMs and human knowledge augmentation
  3. 2025ReviewAgents: Bridging the Gap Between Human and AI-Generated Paper Reviews
  4. 2026From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent
  5. 2025Exploring Design of Multi-Agent LLM Dialogues for Research Ideation
  6. 2025Dynamic Knowledge Exchange and Dual-diversity Review: Concisely Unleashing the Potential of a Multi-Agent Research Team
  7. 2024Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System
  8. 2025LiRA: A Multi-Agent Framework for Reliable and Readable Literature Review Generation