# SurveyForge: On the Outline Heuristics, Memory-Driven Generation, and Multi-dimensional Evaluation for Automated Survey Writing

arXiv: https://arxiv.org/abs/2503.04629 · HTML: https://mlsanity.com/paper/2503.04629/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery](https://mlsanity.com/paper/2410.05080/) (2024)
- [ChartX & ChartVLM: A Versatile Benchmark and Foundation Model for Complicated Chart Reasoning](https://mlsanity.com/paper/2402.12185/) (2024)
- [SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning](https://mlsanity.com/paper/2409.05556/) (2024)
- [OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs](https://mlsanity.com/paper/2411.14199/) (2024)
- [Nova: An Iterative Planning and Search Approach to Enhance Novelty and Diversity of LLM Generated Ideas](https://mlsanity.com/paper/2410.14255/) (2024)
- [SciPIP: An LLM-based Scientific Paper Idea Proposer](https://mlsanity.com/paper/2410.23166/) (2024)
- [Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback](https://mlsanity.com/paper/2501.03916/) (2025)
- [Hierarchical Catalogue Generation for Literature Review: A Benchmark](https://mlsanity.com/paper/2304.03512/) (2023)

## Built on by (most cited first, 30 of 49)

- [KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems](https://mlsanity.com/paper/2510.12872/) (2025)
- [ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks](https://mlsanity.com/paper/2508.15804/) (2025)
- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [Ai2 Scholar QA: Organized Literature Synthesis with Attribution](https://mlsanity.com/paper/2504.10861/) (2025)
- [A Vision for Auto Research with LLM Agents](https://mlsanity.com/paper/2504.18765/) (2025)
- [SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation](https://mlsanity.com/paper/2508.15658/) (2025)
- [SciSage: A Multi-Agent Framework for High-Quality Scientific Survey Generation](https://mlsanity.com/paper/2506.12689/) (2025)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [Context-Aware Hierarchical Taxonomy Generation for Scientific Papers via LLM-Guided Multi-Aspect Clustering](https://mlsanity.com/paper/2509.19125/) (2025)
- [SurveyG: A Multi-Agent LLM Framework with Hierarchical Citation Graph for Automated Survey Generation](https://mlsanity.com/paper/2510.07733/) (2025)
- [Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator](https://mlsanity.com/paper/2507.11810/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Stop DDoS Attacking the Research Community with AI-Generated Survey Papers](https://mlsanity.com/paper/2510.09686/) (2025)
- [Deep Literature Survey Automation with an Iterative Workflow](https://mlsanity.com/paper/2510.21900/) (2025)
- [SurveyGen-I: Consistent Scientific Survey Generation with Evolving Plans and Memory-Guided Writing](https://mlsanity.com/paper/2508.14317/) (2025)
- [SurveyBench: Can LLM(-Agents) Write Academic Surveys that Align with Reader Needs?](https://mlsanity.com/paper/2510.03120/) (2025)
- [Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration](https://mlsanity.com/paper/2505.21471/) (2025)
- [DeepSurvey-Bench: Evaluating Academic Value of Automatically Generated Scientific Surveys](https://mlsanity.com/paper/2601.15307/) (2026)
- [STRUCTSURVEY: Structured Agentic Retrieval for Automated Survey Paper Generation](https://mlsanity.com/paper/2607.01243/) (2026)
- [SGSimEval: A Comprehensive Multifaceted and Similarity-Enhanced Benchmark for Automatic Survey Generation Systems](https://mlsanity.com/paper/2508.11310/) (2025)
- [ARISE: Agentic Rubric-Guided Iterative Survey Engine for Automated Scholarly Paper Generation](https://mlsanity.com/paper/2511.17689/) (2025)
- [Can Deep Research Agents Retrieve and Organize? Evaluating the Synthesis Gap with Expert Taxonomies](https://mlsanity.com/paper/2601.12369/) (2026)
- [Agentic-Ideation: Sample Efficient Agentic Trajectories Synthesis for Scientific Ideation Agents](https://mlsanity.com/paper/2606.31229/) (2026)
- [TaxoAlign: Scholarly Taxonomy Generation Using Language Models](https://mlsanity.com/paper/2510.17263/) (2025)
- [Agentic AI-Empowered Dynamic Survey Framework](https://mlsanity.com/paper/2602.04071/) (2026)
- [Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users](https://mlsanity.com/paper/2603.16120/) (2026)
- [SurveyEval: Towards Comprehensive Evaluation of LLM-Generated Academic Surveys](https://mlsanity.com/paper/2512.02763/) (2025)
- [Eliminating Agentic Workflow for Introduction Generation with Parametric Stage Tokens](https://mlsanity.com/paper/2601.09728/) (2025)
- [HybridQuestion: Human-AI Collaboration for Identifying High-Impact Research Questions](https://mlsanity.com/paper/2602.03849/) (2025)
- [AgentCPM-Report: Interleaving Drafting and Deepening for Open-Ended Deep Research](https://mlsanity.com/paper/2602.06540/) (2026)

## Abstract (arXiv)

Survey paper plays a crucial role in scientific research, especially given the rapid growth of research publications. Recently, researchers have begun using LLMs to automate survey generation for better efficiency. However, the quality gap between LLM-generated surveys and those written by human remains significant, particularly in terms of outline quality and citation accuracy. To close these gaps, we introduce SurveyForge, which first generates the outline by analyzing the logical structure of human-written outlines and referring to the retrieved domain-related articles. Subsequently, leveraging high-quality papers retrieved from memory by our scholar navigation agent, SurveyForge can automatically generate and refine the content of the generated article. Moreover, to achieve a comprehensive evaluation, we construct SurveyBench, which includes 100 human-written survey papers for win-rate comparison and assesses AI-generated survey papers across three dimensions: reference, outline, and content quality. Experiments demonstrate that SurveyForge can outperform previous works such as AutoSurvey.