# ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models

arXiv: https://arxiv.org/abs/2404.07738 · HTML: https://mlsanity.com/paper/2404.07738/

## Key facts

- Cited by 210 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering](https://mlsanity.com/paper/2306.04136/) (2023)
- [The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4](https://mlsanity.com/paper/2311.07361/) (2023)
- [Large Language Models for Automated Open-domain Scientific Hypotheses Discovery](https://mlsanity.com/paper/2309.02726/) (2023)
- [Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering](https://mlsanity.com/paper/2309.11206/) (2023)
- [Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestion](https://mlsanity.com/paper/2311.06318/) (2023)
- [CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent](https://mlsanity.com/paper/2310.06155/) (2023)
- [The ART of LLM Refinement: Ask, Refine, and Trust](https://mlsanity.com/paper/2311.07961/) (2023)
- [Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study](https://mlsanity.com/paper/2106.09700/) (2021)
- [A Computational Inflection for Scientific Discovery](https://mlsanity.com/paper/2205.02007/) (2022)
- [Over-Optimization of Academic Publishing Metrics: Observing Goodhart's Law in Action](https://mlsanity.com/paper/1809.07841/) (2018)
- [AGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approach](https://mlsanity.com/paper/2002.05635/) (2020)
- [We are Who We Cite: Bridges of Influence Between Natural Language Processing and Other Academic Fields](https://mlsanity.com/paper/2310.14870/) (2023)

## Built on by (most cited first, 30 of 210)

- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review](https://mlsanity.com/paper/2504.19678/) (2025)
- [CycleResearcher: Improving Automated Research via Automated Review](https://mlsanity.com/paper/2411.00816/) (2024)
- [MLGym: A New Framework and Benchmark for Advancing AI Research Agents](https://mlsanity.com/paper/2502.14499/) (2025)
- [AI-Researcher: Autonomous Scientific Innovation](https://mlsanity.com/paper/2505.18705/) (2025)
- [Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents](https://mlsanity.com/paper/2410.13185/) (2024)
- [AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML](https://mlsanity.com/paper/2410.02958/) (2024)
- [From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents](https://mlsanity.com/paper/2412.03563/) (2024)
- [Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2503.08979/) (2025)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses](https://mlsanity.com/paper/2410.07076/) (2024)
- [AgentRxiv: Towards Collaborative Autonomous Research](https://mlsanity.com/paper/2503.18102/) (2025)
- [Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning](https://mlsanity.com/paper/2504.17192/) (2025)
- [MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research](https://mlsanity.com/paper/2505.19955/) (2025)
- [DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively](https://mlsanity.com/paper/2509.26603/) (2025)
- [From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents](https://mlsanity.com/paper/2506.18959/) (2025)
- [Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections](https://mlsanity.com/paper/2602.15654/) (2026)
- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [Automated Hypothesis Validation with Agentic Sequential Falsifications](https://mlsanity.com/paper/2502.09858/) (2025)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models](https://mlsanity.com/paper/2405.14767/) (2024)
- [ResearchTown: Simulator of Human Research Community](https://mlsanity.com/paper/2412.17767/) (2024)
- [HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation](https://mlsanity.com/paper/2504.11524/) (2025)
- [EXP-Bench: Can AI Conduct AI Research Experiments?](https://mlsanity.com/paper/2505.24785/) (2025)
- [AutoDiscovery: Open-ended Scientific Discovery via Bayesian Surprise](https://mlsanity.com/paper/2507.00310/) (2025)
- [AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage](https://mlsanity.com/paper/2505.20662/) (2025)

## Abstract (arXiv)

The pace of scientific research, vital for improving human life, is complex, slow, and needs specialized expertise. Meanwhile, novel, impactful research often stems from both a deep understanding of prior work, and a cross-pollination of ideas across domains and fields. To enhance the productivity of researchers, we propose ResearchAgent, which leverages the encyclopedic knowledge and linguistic reasoning capabilities of Large Language Models (LLMs) to assist them in their work. This system automatically defines novel problems, proposes methods and designs experiments, while iteratively refining them based on the feedback from collaborative LLM-powered reviewing agents. Specifically, starting with a core scientific paper, ResearchAgent is augmented not only with relevant publications by connecting information over an academic graph but also entities retrieved from a knowledge store derived from shared underlying concepts mined across numerous papers. Then, mimicking a scientific approach to improving ideas with peer discussions, we leverage multiple LLM-based ReviewingAgents that provide reviews and feedback via iterative revision processes. These reviewing agents are instantiated with human preference-aligned LLMs whose criteria for evaluation are elicited from actual human judgments via LLM prompting. We experimentally validate our ResearchAgent on scientific publications across multiple disciplines, showing its effectiveness in generating novel, clear, and valid ideas based on both human and model-based evaluation results. Our initial foray into AI-mediated scientific research has important implications for the development of future systems aimed at supporting researchers in their ideation and operationalization of novel work.