# CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision

arXiv: https://arxiv.org/abs/2205.06207 · HTML: https://mlsanity.com/paper/2205.06207/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 12 May 2022.

## Builds on (most cited first, 28 of 28)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [A Neural Attention Model for Abstractive Sentence Summarization](https://mlsanity.com/paper/1509.00685/) (2015)
- [Extractive Summarization as Text Matching](https://mlsanity.com/paper/2004.08795/) (2020)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [Structural Scaffolds for Citation Intent Classification in Scientific Publications](https://mlsanity.com/paper/1904.01608/) (2019)
- [Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles](https://mlsanity.com/paper/2010.14235/) (2020)
- [ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks](https://mlsanity.com/paper/1909.01716/) (2019)
- [Sentence Centrality Revisited for Unsupervised Summarization](https://mlsanity.com/paper/1906.03508/) (2019)
- [Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation](https://mlsanity.com/paper/2010.12836/) (2020)
- [A Web-scale system for scientific knowledge exploration](https://mlsanity.com/paper/1805.12216/) (2018)
- [Generating Representative Headlines for News Stories](https://mlsanity.com/paper/2001.09386/) (2020)
- [Explaining Relationships Between Scientific Documents](https://mlsanity.com/paper/2002.00317/) (2020)
- [TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising](https://mlsanity.com/paper/2001.00725/) (2020)
- [Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning](https://mlsanity.com/paper/2010.00117/) (2020)
- [Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering](https://mlsanity.com/paper/2101.00294/) (2021)
- [SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression](https://mlsanity.com/paper/1904.03651/) (2019)
- [TalkSumm: A Dataset and Scalable Annotation Method for Scientific Paper Summarization Based on Conference Talks](https://mlsanity.com/paper/1906.01351/) (2019)
- [Scientific Article Summarization Using Citation-Context and Article's Discourse Structure](https://mlsanity.com/paper/1704.06619/) (2017)
- [Learning to Encode Text as Human-Readable Summaries using Generative Adversarial Networks](https://mlsanity.com/paper/1810.02851/) (2018)
- [Generating Extractive Summaries of Scientific Paradigms](https://mlsanity.com/paper/1402.0556/) (2014)
- [Facet-Aware Evaluation for Extractive Summarization](https://mlsanity.com/paper/1908.10383/) (2019)

## Built on by (most cited first, 15 of 15)

- [What Can Natural Language Processing Do for Peer Review?](https://mlsanity.com/paper/2405.06563/) (2024)
- [unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network](https://mlsanity.com/paper/2303.14957/) (2023)
- [LimGen: Probing the LLMs for Generating Suggestive Limitations of Research Papers](https://mlsanity.com/paper/2403.15529/) (2024)
- [SciNews: From Scholarly Complexities to Public Narratives -- A Dataset for Scientific News Report Generation](https://mlsanity.com/paper/2403.17768/) (2024)
- [Fusing Multimodal Signals on Hyper-complex Space for Extreme Abstractive Text Summarization (TL;DR) of Scientific Contents](https://mlsanity.com/paper/2306.13968/) (2023)
- [When Large Language Models Meet Citation: A Survey](https://mlsanity.com/paper/2309.09727/) (2023)
- [Citance-Contextualized Summarization of Scientific Papers](https://mlsanity.com/paper/2311.02408/) (2023)
- [Unsupervised Domain Adaptation for Keyphrase Generation using Citation Contexts](https://mlsanity.com/paper/2409.13266/) (2024)
- [NUTSHELL: A Dataset for Abstract Generation from Scientific Talks](https://mlsanity.com/paper/2502.16942/) (2025)
- [Surveying the Landscape of Text Summarization with Deep Learning: A Comprehensive Review](https://mlsanity.com/paper/2310.09411/) (2023)
- [CORE: A Few-Shot Company Relation Classification Dataset for Robust Domain Adaptation](https://mlsanity.com/paper/2310.12024/) (2023)
- [Lay Text Summarisation Using Natural Language Processing: A Narrative Literature Review](https://mlsanity.com/paper/2303.14222/) (2023)
- [Introducing Spotlight: A Novel Approach for Generating Captivating Key Information from Documents](https://mlsanity.com/paper/2509.10935/) (2025)
- [Not too long do read: Evaluating LLM-generated extreme scientific summaries](https://mlsanity.com/paper/2512.23206/) (2025)
- [SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era](https://mlsanity.com/paper/2603.16131/) (2026)

## Abstract (arXiv)

Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers. Previous efforts on curating scientific TLDR datasets failed to scale up due to the heavy human annotation and domain expertise required. In this paper, we propose a simple yet effective approach to automatically extracting TLDR summaries for scientific papers from their citation texts. Based on the proposed approach, we create a new benchmark CiteSum without human annotation, which is around 30 times larger than the previous human-curated dataset SciTLDR. We conduct a comprehensive analysis of CiteSum, examining its data characteristics and establishing strong baselines. We further demonstrate the usefulness of CiteSum by adapting models pre-trained on CiteSum (named CITES) to new tasks and domains with limited supervision. For scientific extreme summarization, CITES outperforms most fully-supervised methods on SciTLDR without any fine-tuning and obtains state-of-the-art results with only 128 examples. For news extreme summarization, CITES achieves significant gains on XSum over its base model (not pre-trained on CiteSum), e.g., +7.2 ROUGE-1 zero-shot performance and state-of-the-art few-shot performance. For news headline generation, CITES performs the best among unsupervised and zero-shot methods on Gigaword. Our dataset and code can be found at https://github.com/morningmoni/CiteSum.