# ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary

arXiv: https://arxiv.org/abs/2403.02574 · HTML: https://mlsanity.com/paper/2403.02574/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 5 Mar 2024.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [LitLLM: A Toolkit for Scientific Literature Review](https://mlsanity.com/paper/2402.01788/) (2024)
- [Target-aware Abstractive Related Work Generation with Contrastive Learning](https://mlsanity.com/paper/2205.13339/) (2022)

## Built on by (most cited first, 30 of 30)

- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation](https://mlsanity.com/paper/2502.05151/) (2025)
- [A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval](https://mlsanity.com/paper/2503.05659/) (2025)
- [AIGS: Generating Science from AI-Powered Automated Falsification](https://mlsanity.com/paper/2411.11910/) (2024)
- [MemGuide: Intent-Driven Memory Selection for Goal-Oriented Multi-Session LLM Agents](https://mlsanity.com/paper/2505.20231/) (2025)
- [DeepSeek vs. ChatGPT vs. Claude: A Comparative Study for Scientific Computing and Scientific Machine Learning Tasks](https://mlsanity.com/paper/2502.17764/) (2025)
- [InteractiveSurvey: An LLM-based Personalized and Interactive Survey Paper Generation System](https://mlsanity.com/paper/2504.08762/) (2025)
- [BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Science](https://mlsanity.com/paper/2407.00466/) (2024)
- [Evolving Roles of LLMs in Scientific Innovation: Assistant, Collaborator, Scientist, and Evaluator](https://mlsanity.com/paper/2507.11810/) (2025)
- [Automated Review Generation Method Based on Large Language Models](https://mlsanity.com/paper/2407.20906/) (2024)
- [Cutting Through the Clutter: The Potential of LLMs for Efficient Filtration in Systematic Literature Reviews](https://mlsanity.com/paper/2407.10652/) (2024)
- [Exploring Prompt Engineering: A Systematic Review with SWOT Analysis](https://mlsanity.com/paper/2410.12843/) (2024)
- [Completing A Systematic Review in Hours instead of Months with Interactive AI Agents](https://mlsanity.com/paper/2504.14822/) (2025)
- [SurveyGen-I: Consistent Scientific Survey Generation with Evolving Plans and Memory-Guided Writing](https://mlsanity.com/paper/2508.14317/) (2025)
- [GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Serving](https://mlsanity.com/paper/2605.22566/) (2026)
- [Iter-AHMCL: Alleviate Hallucination for Large Language Model via Iterative Model-level Contrastive Learning](https://mlsanity.com/paper/2410.12130/) (2024)
- [Modelling and Classifying the Components of a Literature Review](https://mlsanity.com/paper/2508.04337/) (2025)
- [ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference](https://mlsanity.com/paper/2509.03565/) (2025)
- [Adaptive and Resource-efficient Agentic AI Systems for Mobile and Embedded Devices: A Survey](https://mlsanity.com/paper/2510.00078/) (2025)
- [pAI/MSc: ML Theory Research with Humans on the Loop](https://mlsanity.com/paper/2604.20622/) (2026)
- [Multi-Turn Agentic Scientific Literature Search via Workflow Induction](https://mlsanity.com/paper/2607.00597/) (2026)
- [Highlighting Case Studies in LLM Literature Review of Interdisciplinary System Science](https://mlsanity.com/paper/2503.16515/) (2025)
- [EvoVLMA: Evolutionary Vision-Language Model Adaptation](https://mlsanity.com/paper/2508.01558/) (2025)
- [Leveraging LLMs for Semi-Automatic Corpus Filtration in Systematic Literature Reviews](https://mlsanity.com/paper/2510.11409/) (2025)
- [From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews](https://mlsanity.com/paper/2512.11661/) (2025)
- [Leveraging Large Language Models for Comparative Literature Summarization with Reflective Incremental Mechanisms](https://mlsanity.com/paper/2412.02149/) (2024)
- [CL2GEC: A Multi-Discipline Benchmark for Continual Learning in Chinese Literature Grammatical Error Correction](https://mlsanity.com/paper/2509.13672/) (2025)
- [An Artificial Intelligence Driven Semantic Similarity-Based Pipeline for Rapid Literature](https://mlsanity.com/paper/2509.15292/) (2025)
- [Remembering Unequally: Global and Disciplinary Bias in LLM Reconstruction of Scholarly Coauthor Lists](https://mlsanity.com/paper/2511.00476/) (2025)
- [AgenticSum: An Agentic Inference-Time Framework for Faithful Clinical Text Summarization](https://mlsanity.com/paper/2602.20040/) (2026)

## Abstract (arXiv)

The literature review is an indispensable step in the research process. It provides the benefit of comprehending the research problem and understanding the current research situation while conducting a comparative analysis of prior works. However, literature summary is challenging and time consuming. The previous LLM-based studies on literature review mainly focused on the complete process, including literature retrieval, screening, and summarization. However, for the summarization step, simple CoT method often lacks the ability to provide extensive comparative summary. In this work, we firstly focus on the independent literature summarization step and introduce ChatCite, an LLM agent with human workflow guidance for comparative literature summary. This agent, by mimicking the human workflow, first extracts key elements from relevant literature and then generates summaries using a Reflective Incremental Mechanism. In order to better evaluate the quality of the generated summaries, we devised a LLM-based automatic evaluation metric, G-Score, in refer to the human evaluation criteria. The ChatCite agent outperformed other models in various dimensions in the experiments. The literature summaries generated by ChatCite can also be directly used for drafting literature reviews.