# Leveraging Large Language Models for Comparative Literature Summarization with Reflective Incremental Mechanisms

arXiv: https://arxiv.org/abs/2412.02149 · HTML: https://mlsanity.com/paper/2412.02149/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 3 Dec 2024.

## Builds on (most cited first, 20 of 20)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [LooGLE: Can Long-Context Language Models Understand Long Contexts?](https://mlsanity.com/paper/2311.04939/) (2023)
- [Visual In-Context Learning for Large Vision-Language Models](https://mlsanity.com/paper/2402.11574/) (2024)
- [Retrieval meets Long Context Large Language Models](https://mlsanity.com/paper/2310.03025/) (2023)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models](https://mlsanity.com/paper/2401.03205/) (2024)
- [Thread of Thought Unraveling Chaotic Contexts](https://mlsanity.com/paper/2311.08734/) (2023)
- [LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models](https://mlsanity.com/paper/2307.07889/) (2023)
- [LitLLM: A Toolkit for Scientific Literature Review](https://mlsanity.com/paper/2402.01788/) (2024)
- [BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models](https://mlsanity.com/paper/2309.13345/) (2023)
- [ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification](https://mlsanity.com/paper/2203.02225/) (2022)
- [Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Models](https://mlsanity.com/paper/2410.19732/) (2024)
- [EventBERT: A Pre-Trained Model for Event Correlation Reasoning](https://mlsanity.com/paper/2110.06533/) (2021)
- [Towards Robust Ranker for Text Retrieval](https://mlsanity.com/paper/2206.08063/) (2022)
- [ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary](https://mlsanity.com/paper/2403.02574/) (2024)
- [Fine-Grained Distillation for Long Document Retrieval](https://mlsanity.com/paper/2212.10423/) (2022)
- [Content Reduction, Surprisal and Information Density Estimation for Long Documents](https://mlsanity.com/paper/2309.06009/) (2023)

## Abstract (arXiv)

In this paper, we introduce ChatCite, a novel method leveraging large language models (LLMs) for generating comparative literature summaries. The ability to summarize research papers with a focus on key comparisons between studies is an essential task in academic research. Existing summarization models, while effective at generating concise summaries, fail to provide deep comparative insights. ChatCite addresses this limitation by incorporating a multi-step reasoning mechanism that extracts critical elements from papers, incrementally builds a comparative summary, and refines the output through a reflective memory process. We evaluate ChatCite on a custom dataset, CompLit-LongContext, consisting of 1000 research papers with annotated comparative summaries. Experimental results show that ChatCite outperforms several baseline methods, including GPT-4, BART, T5, and CoT, across various automatic evaluation metrics such as ROUGE and the newly proposed G-Score. Human evaluation further confirms that ChatCite generates more coherent, insightful, and fluent summaries compared to these baseline models. Our method provides a significant advancement in automatic literature review generation, offering researchers a powerful tool for efficiently comparing and synthesizing scientific research.