# From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting

arXiv: https://arxiv.org/abs/2309.04269 · HTML: https://mlsanity.com/paper/2309.04269/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 8 Sep 2023.

## Builds on (most cited first, 28 of 28)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://mlsanity.com/paper/1804.11283/) (2018)
- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation](https://mlsanity.com/paper/2212.07981/) (2022)
- [GSum: A General Framework for Guided Neural Abstractive Summarization](https://mlsanity.com/paper/2010.08014/) (2020)
- [Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2109.09784/) (2021)
- [Entity-level Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/2102.09130/) (2021)
- [CTRLsum: Towards Generic Controllable Text Summarization](https://mlsanity.com/paper/2012.04281/) (2020)
- [Planning with Learned Entity Prompts for Abstractive Summarization](https://mlsanity.com/paper/2104.07606/) (2021)
- [Neural Document Summarization by Jointly Learning to Score and Select Sentences](https://mlsanity.com/paper/1807.02305/) (2018)
- [ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer](https://mlsanity.com/paper/2306.07799/) (2023)
- [Prompted Opinion Summarization with GPT-3.5](https://mlsanity.com/paper/2211.15914/) (2022)
- [mFACE: Multilingual Summarization with Factual Consistency Evaluation](https://mlsanity.com/paper/2212.10622/) (2022)
- [Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics](https://mlsanity.com/paper/2204.10216/) (2022)
- [Controllable Neural Dialogue Summarization with Personal Named Entity Planning](https://mlsanity.com/paper/2109.13070/) (2021)
- [Learning to Revise References for Faithful Summarization](https://mlsanity.com/paper/2204.10290/) (2022)
- [EntSUM: A Data Set for Entity-Centric Summarization](https://mlsanity.com/paper/2204.02213/) (2022)
- [A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarization](https://mlsanity.com/paper/2303.03948/) (2023)
- [Generating EDU Extracts for Plan-Guided Summary Re-Ranking](https://mlsanity.com/paper/2305.17779/) (2023)

## Built on by (most cited first, 30 of 52)

- [BooookScore: A systematic exploration of book-length summarization in the era of LLMs](https://mlsanity.com/paper/2310.00785/) (2023)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [Unleashing the potential of prompt engineering for large language models](https://mlsanity.com/paper/2310.14735/) (2023)
- [Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization](https://mlsanity.com/paper/2311.09184/) (2023)
- [Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Models](https://mlsanity.com/paper/2407.16221/) (2024)
- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality](https://mlsanity.com/paper/2505.18227/) (2025)
- [Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs](https://mlsanity.com/paper/2404.15676/) (2024)
- [On Context Utilization in Summarization with Large Language Models](https://mlsanity.com/paper/2310.10570/) (2023)
- [Image Clustering Conditioned on Text Criteria](https://mlsanity.com/paper/2310.18297/) (2023)
- [RAD-Bench: Evaluating Large Language Models Capabilities in Retrieval Augmented Dialogues](https://mlsanity.com/paper/2409.12558/) (2024)
- [Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization](https://mlsanity.com/paper/2311.09335/) (2023)
- [GINGER: Grounded Information Nugget-Based Generation of Responses](https://mlsanity.com/paper/2503.18174/) (2025)
- [LLM-RadJudge: Achieving Radiologist-Level Evaluation for X-Ray Report Generation](https://mlsanity.com/paper/2404.00998/) (2024)
- [Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data](https://mlsanity.com/paper/2410.18588/) (2024)
- [Generating Educational Materials with Different Levels of Readability using LLMs](https://mlsanity.com/paper/2406.12787/) (2024)
- [GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization](https://mlsanity.com/paper/2410.04087/) (2024)
- [QuaLLM: An LLM-based Framework to Extract Quantitative Insights from Online Forums](https://mlsanity.com/paper/2405.05345/) (2024)
- [SPeCtrum: A Grounded Framework for Multidimensional Identity Representation in LLM-Based Agent](https://mlsanity.com/paper/2502.08599/) (2025)
- [Large Language Models are Interpretable Learners](https://mlsanity.com/paper/2406.17224/) (2024)
- [Prompted Aspect Key Point Analysis for Quantitative Review Summarization](https://mlsanity.com/paper/2407.14049/) (2024)
- [SPEER: Sentence-Level Planning of Long Clinical Summaries via Embedded Entity Retrieval](https://mlsanity.com/paper/2401.02369/) (2024)
- [Bridging Research and Readers: A Multi-Modal Automated Academic Papers Interpretation System](https://mlsanity.com/paper/2401.09150/) (2024)
- [Multi-LLM Text Summarization](https://mlsanity.com/paper/2412.15487/) (2024)
- [BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Serving](https://mlsanity.com/paper/2507.17120/) (2025)
- [Salient Information Prompting to Steer Content in Prompt-based Abstractive Summarization](https://mlsanity.com/paper/2410.02741/) (2024)
- [Smart Audit System Empowered by LLM](https://mlsanity.com/paper/2410.07677/) (2024)
- [MODS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collections](https://mlsanity.com/paper/2502.00322/) (2025)
- [QA-prompting: Improving Summarization with Large Language Models using Question-Answering](https://mlsanity.com/paper/2505.14347/) (2025)

## Abstract (arXiv)

Selecting the ``right'' amount of information to include in a summary is a difficult task. A good summary should be detailed and entity-centric without being overly dense and hard to follow. To better understand this tradeoff, we solicit increasingly dense GPT-4 summaries with what we refer to as a ``Chain of Density'' (CoD) prompt. Specifically, GPT-4 generates an initial entity-sparse summary before iteratively incorporating missing salient entities without increasing the length. Summaries generated by CoD are more abstractive, exhibit more fusion, and have less of a lead bias than GPT-4 summaries generated by a vanilla prompt. We conduct a human preference study on 100 CNN DailyMail articles and find that that humans prefer GPT-4 summaries that are more dense than those generated by a vanilla prompt and almost as dense as human written summaries. Qualitative analysis supports the notion that there exists a tradeoff between informativeness and readability. 500 annotated CoD summaries, as well as an extra 5,000 unannotated summaries, are freely available on HuggingFace (https://huggingface.co/datasets/griffin/chain_of_density).