# Leveraging Information Bottleneck for Scientific Document Summarization

arXiv: https://arxiv.org/abs/2110.01280 · HTML: https://mlsanity.com/paper/2110.01280/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 4 Oct 2021.

## Builds on (most cited first, 25 of 25)

- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Efficient Attentions for Long Document Summarization](https://mlsanity.com/paper/2104.02112/) (2021)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [CORD-19: The COVID-19 Open Research Dataset](https://mlsanity.com/paper/2004.10706/) (2020)
- [GSum: A General Framework for Guided Neural Abstractive Summarization](https://mlsanity.com/paper/2010.08014/) (2020)
- [HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization](https://mlsanity.com/paper/1905.06566/) (2019)
- [Neural Summarization by Extracting Sentences and Words](https://mlsanity.com/paper/1603.07252/) (2016)
- [Extractive Summarization of Long Documents by Combining Global and Local Context](https://mlsanity.com/paper/1909.08089/) (2019)
- [BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle](https://mlsanity.com/paper/1909.07405/) (2019)
- [Variations of the Similarity Function of TextRank for Automated Summarization](https://mlsanity.com/paper/1602.03606/) (2016)
- [Sample Efficient Text Summarization Using a Single Pre-Trained Transformer](https://mlsanity.com/paper/1905.08836/) (2019)
- [An Entity-Driven Framework for Abstractive Summarization](https://mlsanity.com/paper/1909.02059/) (2019)
- [SummPip: Unsupervised Multi-Document Summarization with Sentence Graph Compression](https://mlsanity.com/paper/2007.08954/) (2020)
- [Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks](https://mlsanity.com/paper/2010.06253/) (2020)
- [Abstractive Summarization with Combination of Pre-trained Sequence-to-Sequence and Saliency Models](https://mlsanity.com/paper/2003.13028/) (2020)
- [Phase Transitions for the Information Bottleneck in Representation Learning](https://mlsanity.com/paper/2001.01878/) (2020)
- [Enhancing Factual Consistency of Abstractive Summarization](https://mlsanity.com/paper/2003.08612/) (2020)
- [SciSummPip: An Unsupervised Scientific Paper Summarization Pipeline](https://mlsanity.com/paper/2010.09190/) (2020)

## Built on by (most cited first, 13 of 13)

- [LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization](https://mlsanity.com/paper/2301.13298/) (2023)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding](https://mlsanity.com/paper/2306.04933/) (2023)
- [HEGEL: Hypergraph Transformer for Long Document Summarization](https://mlsanity.com/paper/2210.04126/) (2022)
- [How Far are We from Robust Long Abstractive Summarization?](https://mlsanity.com/paper/2210.16732/) (2022)
- [What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations](https://mlsanity.com/paper/2502.08279/) (2025)
- [PosterSum: A Multimodal Benchmark for Scientific Poster Summarization](https://mlsanity.com/paper/2502.17540/) (2025)
- [Contrastive Hierarchical Discourse Graph for Scientific Document Summarization](https://mlsanity.com/paper/2306.00177/) (2023)
- [BottleHumor: Self-Informed Humor Explanation using the Information Bottleneck Principle](https://mlsanity.com/paper/2502.18331/) (2025)
- [Explanation Regeneration via Information Bottleneck](https://mlsanity.com/paper/2212.09603/) (2022)
- [Summarization with Graphical Elements](https://mlsanity.com/paper/2204.07551/) (2022)
- [Question-Answer Extraction from Scientific Articles Using Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2507.13827/) (2025)
- [Information-Theoretic Complementary Prompts for Improved Continual Text Classification](https://mlsanity.com/paper/2505.20933/) (2025)

## Abstract (arXiv)

This paper presents an unsupervised extractive approach to summarize scientific long documents based on the Information Bottleneck principle. Inspired by previous work which uses the Information Bottleneck principle for sentence compression, we extend it to document level summarization with two separate steps. In the first step, we use signal(s) as queries to retrieve the key content from the source document. Then, a pre-trained language model conducts further sentence search and edit to return the final extracted summaries. Importantly, our work can be flexibly extended to a multi-view framework by different signals. Automatic evaluation on three scientific document datasets verifies the effectiveness of the proposed framework. The further human evaluation suggests that the extracted summaries cover more content aspects than previous systems.