mlsanityData to 14 Oct 2026

From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting

arXiv 2309.04269 ↗First posted 8 Sep 2023cs.CLcited by 52

Not on the map. Why?

Abstract (arXiv)

Selecting the ``right'' amount of information to include in a summary is a difficult task. A good summary should be detailed and entity-centric without being overly dense and hard to follow. To better understand this tradeoff, we solicit increasingly dense GPT-4 summaries with what we refer to as a ``Chain of Density'' (CoD) prompt. Specifically, GPT-4 generates an initial entity-sparse summary before iteratively incorporating missing salient entities without increasing the length. Summaries generated by CoD are more abstractive, exhibit more fusion, and have less of a lead bias than GPT-4 summaries generated by a vanilla prompt. We conduct a human preference study on 100 CNN DailyMail articles and find that that humans prefer GPT-4 summaries that are more dense than those generated by a vanilla prompt and almost as dense as human written summaries. Qualitative analysis supports the notion that there exists a tradeoff between informativeness and readability. 500 annotated CoD summaries, as well as an extra 5,000 unannotated summaries, are freely available on HuggingFace (https://huggingface.co/datasets/griffin/chain_of_density).

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 8 Sep 2023.

Builds on

28 of 34 references.

References · 28

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2020Learning to summarize from human feedbackcited by 2,588
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  5. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  6. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  7. 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
  8. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  9. 2023GPTScore: Evaluate as You Desirecited by 361
  10. 2022News Summarization and Evaluation in the Era of GPT-3cited by 327
  11. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 294
  12. 2023Challenges and Applications of Large Language Modelscited by 254
16 more
  1. 2022Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluationcited by 142
  2. 2020GSum: A General Framework for Guided Neural Abstractive Summarizationcited by 129
  3. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 129
  4. 2021Entity-level Factual Consistency of Abstractive Text Summarizationcited by 118
  5. 2020CTRLsum: Towards Generic Controllable Text Summarizationcited by 98
  6. 2021Planning with Learned Entity Prompts for Abstractive Summarizationcited by 75
  7. 2018Neural Document Summarization by Jointly Learning to Score and Select Sentencescited by 56
  8. 2023ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfercited by 50
  9. 2022Prompted Opinion Summarization with GPT-3.5cited by 44
  10. 2022mFACE: Multilingual Summarization with Factual Consistency Evaluationcited by 42
  11. 2022Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metricscited by 38
  12. 2021Controllable Neural Dialogue Summarization with Personal Named Entity Planningcited by 36
  13. 2022Learning to Revise References for Faithful Summarizationcited by 28
  14. 2022EntSUM: A Data Set for Entity-Centric Summarizationcited by 22
  15. 2023A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarizationcited by 14
  16. 2023Generating EDU Extracts for Plan-Guided Summary Re-Rankingcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 23; 2025: 14; 2026: 8.

52 papers.

Sort:

By area:

  1. 2023BooookScore: A systematic exploration of book-length summarization in the era of LLMscited by 137
  2. 2024Large Language Models Meet NLP: A Surveycited by 103
  3. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 102
  4. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  5. 2023Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarizationcited by 69
  6. 2024Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Modelscited by 57
  7. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  8. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 50
  9. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMscited by 48
  10. 2023On Context Utilization in Summarization with Large Language Modelscited by 30
  11. 2023Image Clustering Conditioned on Text Criteriacited by 26
  12. 2024RAD-Bench: Evaluating Large Language Models Capabilities in Retrieval Augmented Dialoguescited by 19
38 more
  1. 2023Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarizationcited by 16
  2. 2025GINGER: Grounded Information Nugget-Based Generation of Responsescited by 16
  3. 2024LLM-RadJudge: Achieving Radiologist-Level Evaluation for X-Ray Report Generationcited by 15
  4. 2024Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Datacited by 12
  5. 2024Generating Educational Materials with Different Levels of Readability using LLMscited by 9
  6. 2024GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarizationcited by 9
  7. 2024QuaLLM: An LLM-based Framework to Extract Quantitative Insights from Online Forumscited by 8
  8. 2025SPeCtrum: A Grounded Framework for Multidimensional Identity Representation in LLM-Based Agentcited by 8
  9. 2024Large Language Models are Interpretable Learnerscited by 7
  10. 2024Prompted Aspect Key Point Analysis for Quantitative Review Summarizationcited by 6
  11. 2024SPEER: Sentence-Level Planning of Long Clinical Summaries via Embedded Entity Retrievalcited by 5
  12. 2024Bridging Research and Readers: A Multi-Modal Automated Academic Papers Interpretation Systemcited by 5
  13. 2024Multi-LLM Text Summarizationcited by 5
  14. 2025BucketServe: Bucket-Based Dynamic Batching for Smart and Efficient LLM Inference Servingcited by 5
  15. 2024Salient Information Prompting to Steer Content in Prompt-based Abstractive Summarizationcited by 4
  16. 2024Smart Audit System Empowered by LLMcited by 4
  17. 2025MODS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collectionscited by 4
  18. 2025QA-prompting: Improving Summarization with Large Language Models using Question-Answeringcited by 4
  19. 2024Language-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the Videocited by 3
  20. 2024uMedSum: A Unified Framework for Advancing Medical Abstractive Summarizationcited by 3
  21. 2025ARC: Argument Representation and Coverage Analysis for Zero-Shot Long Document Summarization with Instruction Following LLMscited by 3
  22. 2026Federation over Text: Insight Sharing for Multi-Agent Reasoningcited by 3
  23. 2023Novel Preprocessing Technique for Data Embedding in Engineering Code Generation Using Large Language Modelcited by 2
  24. 2025Improving LLM-based Document-level Machine Translation with Multi-Knowledge Fusioncited by 2
  25. 2025REFER: Mitigating Bias in Opinion Summarisation via Frequency Framed Promptingcited by 2
  26. 2024Entity-level Factual Adaptiveness of Fine-tuning based Abstractive Summarization Modelscited by 1
  27. 2024Context-Informed Machine Translation of Manga using Multimodal Large Language Modelscited by 1
  28. 2025Semantic Captioning: Benchmark Dataset and Graph-Aware Few-Shot In-Context Learning for SQL2Textcited by 1
  29. 2025UiS-IAI@LiveRAG: Retrieval-Augmented Information Nugget-Based Generation of Responsescited by 1
  30. 2026BenchBrowser: Retrieving Evidence for Evaluating Benchmark Validitycited by 1
  31. 2024Evaluating GPT-3.5's Awareness and Summarization Abilities for European Constitutional Texts with Shared Topicscited by 0
  32. 2025Chain of Summaries: Summarization Through Iterative Questioningcited by 0
  33. 2025The Effect of Document Summarization on LLM-Based Relevance Judgmentscited by 0
  34. 2025A Dataset and Benchmark for Consumer Healthcare Question Summarizationcited by 0
  35. 2026STELLA: Self-Reflective Terminology-Aware Framework for Building an Aerospace Information Retrieval Benchmarkcited by 0
  36. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0
  37. 2026Generative Artificial Intelligence for Literature Reviewscited by 0
  38. 2026Summarization is Not Dead Yetcited by 0

50 most cited of 52 All 52 (JSON)