# Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System

arXiv: https://arxiv.org/abs/2410.09403 · HTML: https://mlsanity.com/paper/2410.09403/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 83 references that are in the map.
- First posted on arXiv 12 Oct 2024.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 22 of 22)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [DiGress: Discrete Denoising diffusion for graph generation](https://mlsanity.com/paper/2209.14734/) (2022)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View](https://mlsanity.com/paper/2310.02124/) (2023)
- [Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?](https://mlsanity.com/paper/2402.18272/) (2024)
- [AgentScope: A Flexible yet Robust Multi-Agent Platform](https://mlsanity.com/paper/2402.14034/) (2024)
- [Can Large Language Model Agents Simulate Human Trust Behavior?](https://mlsanity.com/paper/2402.04559/) (2024)
- [CRISPR-GPT for Agentic Automation of Gene-editing Experiments](https://mlsanity.com/paper/2404.18021/) (2024)
- [Are LLMs Ready for Real-World Materials Discovery?](https://mlsanity.com/paper/2402.05200/) (2024)
- [Large Language Models for Scientific Synthesis, Inference and Explanation](https://mlsanity.com/paper/2310.07984/) (2023)
- [A Survey of Deep Learning for Scientific Discovery](https://mlsanity.com/paper/2003.11755/) (2020)
- [Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation](https://mlsanity.com/paper/2407.08940/) (2024)
- [Multi-Agent Collaboration via Cross-Team Orchestration](https://mlsanity.com/paper/2406.08979/) (2024)
- [Fine-Grained Distillation for Long Document Retrieval](https://mlsanity.com/paper/2212.10423/) (2022)
- [CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Support](https://mlsanity.com/paper/2407.16148/) (2024)
- [An LLM-based Knowledge Synthesis and Scientific Reasoning Framework for Biomedical Discovery](https://mlsanity.com/paper/2406.18626/) (2024)

## Built on by (most cited first, 30 of 68)

- [EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery](https://mlsanity.com/paper/2603.08127/) (2026)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaboration](https://mlsanity.com/paper/2505.15047/) (2025)
- [Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoning](https://mlsanity.com/paper/2504.05047/) (2025)
- [Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback](https://mlsanity.com/paper/2501.03916/) (2025)
- [Creativity in LLM-based Multi-Agent Systems: A Survey](https://mlsanity.com/paper/2505.21116/) (2025)
- [Literature-Grounded Novelty Assessment of Scientific Ideas](https://mlsanity.com/paper/2506.22026/) (2025)
- [AIGS: Generating Science from AI-Powered Automated Falsification](https://mlsanity.com/paper/2411.11910/) (2024)
- [Generative Adversarial Reviews: When LLMs Become the Critic](https://mlsanity.com/paper/2412.10415/) (2024)
- [Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics](https://mlsanity.com/paper/2510.09901/) (2025)
- [BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation](https://mlsanity.com/paper/2510.10157/) (2025)
- [A Survey of AI Scientists](https://mlsanity.com/paper/2510.23045/) (2025)
- [ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers?](https://mlsanity.com/paper/2510.24591/) (2025)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [Is this Idea Novel? An Automated Benchmark for Judgment of Research Ideas](https://mlsanity.com/paper/2603.10303/) (2026)
- [Measuring the Gap Between Human and LLM Research Ideas](https://mlsanity.com/paper/2607.01233/) (2026)
- [Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator](https://mlsanity.com/paper/2507.11810/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Exploring Design of Multi-Agent LLM Dialogues for Research Ideation](https://mlsanity.com/paper/2507.08350/) (2025)
- [RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse](https://mlsanity.com/paper/2603.13289/) (2026)
- [Deep Ideation: Designing LLM Agents to Generate Novel Research Ideas on Scientific Concept Network](https://mlsanity.com/paper/2511.02238/) (2025)
- [Principle-Evolvable Scientific Discovery via Uncertainty Minimization](https://mlsanity.com/paper/2602.06448/) (2026)
- [NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment](https://mlsanity.com/paper/2604.11543/) (2026)
- [Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generation](https://mlsanity.com/paper/2604.18005/) (2026)
- [Unlocking LLM Creativity in Science through Analogical Reasoning](https://mlsanity.com/paper/2605.11258/) (2026)
- [Can AI Validate Science? Benchmarking LLMs for Accurate Scientific Claim rightarrow Evidence Reasoning](https://mlsanity.com/paper/2506.08235/) (2025)
- [BioVerge: A Comprehensive Benchmark and Study of Self-Evaluating Agents for Biomedical Hypothesis Generation](https://mlsanity.com/paper/2511.08866/) (2025)
- [InnoGym: Benchmarking the Innovation Potential of AI Agents](https://mlsanity.com/paper/2512.01822/) (2025)
- [LDC: Learning to Generate Research Idea with Dynamic Control](https://mlsanity.com/paper/2412.14626/) (2024)
- [Towards Agentic Intelligence for Materials Science](https://mlsanity.com/paper/2602.00169/) (2026)

## Abstract (arXiv)

The rapid advancement of scientific progress requires innovative tools that can accelerate knowledge discovery. Although recent AI methods, particularly large language models (LLMs), have shown promise in tasks such as hypothesis generation and experimental design, they fall short of replicating the collaborative nature of real-world scientific practices, where diverse experts work together in teams to tackle complex problems. To address the limitations, we propose an LLM-based multi-agent system, i.e., Virtual Scientists (VirSci), designed to mimic the teamwork inherent in scientific research. VirSci organizes a team of agents to collaboratively generate, evaluate, and refine research ideas. Through comprehensive experiments, we demonstrate that this multi-agent approach outperforms the state-of-the-art method in producing novel scientific ideas. We further investigate the collaboration mechanisms that contribute to its tendency to produce ideas with higher novelty, offering valuable insights to guide future research and illuminating pathways toward building a robust system for autonomous scientific discovery. The code is available at https://github.com/open-sciencelab/Virtual-Scientists.