# SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization

arXiv: https://arxiv.org/abs/2006.10213 · HTML: https://mlsanity.com/paper/2006.10213/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 15 of 15)

- [Efficient Attentions for Long Document Summarization](https://mlsanity.com/paper/2104.02112/) (2021)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documents](https://mlsanity.com/paper/2110.10150/) (2021)
- [Exploring Neural Models for Query-Focused Summarization](https://mlsanity.com/paper/2112.07637/) (2021)
- [Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents](https://mlsanity.com/paper/2106.00130/) (2021)
- [How Far are We from Robust Long Abstractive Summarization?](https://mlsanity.com/paper/2210.16732/) (2022)
- [Summarization Programs: Interpretable Abstractive Summarization with Neural Modular Trees](https://mlsanity.com/paper/2209.10492/) (2022)
- [Leveraging Locality in Abstractive Text Summarization](https://mlsanity.com/paper/2205.12476/) (2022)
- [The Current State of Summarization](https://mlsanity.com/paper/2305.04853/) (2023)
- [EASE: Extractive-Abstractive Summarization with Explanations](https://mlsanity.com/paper/2105.06982/) (2021)
- [MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization](https://mlsanity.com/paper/2109.10650/) (2021)
- [AWESOME: GPU Memory-constrained Long Document Summarization using Memory Mechanism and Global Salient Content](https://mlsanity.com/paper/2305.14806/) (2023)
- [Learning to Summarize by Learning to Quiz: Adversarial Agentic Collaboration for Long Document Summarization](https://mlsanity.com/paper/2509.20900/) (2025)

## Abstract (arXiv)

Most prior work in the sequence-to-sequence paradigm focused on datasets with input sequence lengths in the hundreds of tokens due to the computational constraints of common RNN and Transformer architectures. In this paper, we study long-form abstractive text summarization, a sequence-to-sequence setting with input sequence lengths up to 100,000 tokens and output sequence lengths up to 768 tokens. We propose SEAL, a Transformer-based model, featuring a new encoder-decoder attention that dynamically extracts/selects input snippets to sparsely attend to for each output segment. Using only the original documents and summaries, we derive proxy labels that provide weak supervision for extractive layers simultaneously with regular supervision from abstractive summaries. The SEAL model achieves state-of-the-art results on existing long-form summarization tasks, and outperforms strong baseline models on a new dataset/task we introduce, Search2Wiki, with much longer input text. Since content selection is explicit in the SEAL model, a desirable side effect is that the selection can be inspected for enhanced interpretability.