mlsanityData to 7 Oct 2026

MemSum: Extractive Summarization of Long Documents Using Multi-Step Episodic Markov Decision Processes

arXiv 2107.08929 ↗First posted 19 Jul 2021cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

We introduce MemSum (Multi-step Episodic Markov decision process extractive SUMmarizer), a reinforcement-learning-based extractive summarizer enriched at each step with information on the current extraction history. When MemSum iteratively selects sentences into the summary, it considers a broad information set that would intuitively also be used by humans in this task: 1) the text content of the sentence, 2) the global text context of the rest of the document, and 3) the extraction history consisting of the set of sentences that have already been extracted. With a lightweight architecture, MemSum obtains state-of-the-art test-set performance (ROUGE) in summarizing long documents taken from PubMed, arXiv, and GovReport. Ablation studies demonstrate the importance of local, global, and history information. A human evaluation confirms the high quality and low redundancy of the generated summaries, stemming from MemSum's awareness of extraction history.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 28 of its 46 references that are in the map. First posted on arXiv 19 Jul 2021.

Builds on

28 of 46 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  8. 2020Reformer: The Efficient Transformercited by 1,658
  9. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  10. 2015Teaching Machines to Read and Comprehendcited by 1,039
  11. 2019Text Summarization with Pretrained Encoderscited by 553
  12. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
16 more
  1. 2021Efficient Attentions for Long Document Summarizationcited by 342
  2. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  3. 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 245
  4. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  5. 2020Extractive Summarization as Text Matchingcited by 189
  6. 2019Hierarchical Transformers for Multi-Document Summarizationcited by 132
  7. 2018Ranking Sentences for Extractive Summarization with Reinforcement Learningcited by 128
  8. 2019HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarizationcited by 120
  9. 2017Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Featurescited by 103
  10. 2019On Extractive and Abstractive Neural Document Summarization with Transformer Language Modelscited by 83
  11. 2018Learning to Extract Coherent Summary via Deep Reinforcement Learningcited by 63
  12. 2019Extractive Summarization of Long Documents by Combining Global and Local Contextcited by 59
  13. 2018Neural Document Summarization by Jointly Learning to Score and Select Sentencescited by 56
  14. 2018BanditSum: Extractive Summarization as a Contextual Banditcited by 46
  15. 2018A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)cited by 9
  16. 2019Improving Abstractive Text Summarization with History Aggregationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 4; 2025: 2; 2026: 3.

19 papers.

Sort:

By area:

  1. 2022Prompted Opinion Summarization with GPT-3.5cited by 44
  2. 2023Scientific Opinion Summarization: Paper Meta-review Generation Dataset, Methods, and Evaluationcited by 14
  3. 2026Text summarization via global structure awarenesscited by 11
  4. 2022GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse statecited by 9
  5. 2023Legal Extractive Summarization of U.S. Court Opinionscited by 9
  6. 2023Cross-lingual Cross-temporal Summarization: Dataset, Models, Evaluationcited by 8
  7. 2023Hybrid Long Document Summarization using C2F-FAR and ChatGPT: A Practical Studycited by 5
  8. 2023SciLit: A Platform for Joint Scientific Literature Discovery, Summarization and Citation Generationcited by 5
  9. 2024PostDoc: Generating Poster from a Long Multimodal Document Using Deep Submodular Optimizationcited by 5
  10. 2024Incremental Extractive Opinion Summarization Using Cover Treescited by 4
  11. 2024Hierarchical Attention Graph for Scientific Document Summarization in Global and Local Levelcited by 4
  12. 2022EDU-level Extractive Summarization with Varying Summary Lengthscited by 3
7 more
  1. 2023Enhancing Coherence of Extractive Summarization with Multitask Learningcited by 1
  2. 2023MemSum-DQA: Adapting An Efficient Long Document Extractive Summarizer for Document Question Answeringcited by 1
  3. 2025A Cascaded Architecture for Extractive Summarization of Multimedia Content via Audio-to-Text Alignmentcited by 1
  4. 2026From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priorscited by 1
  5. 2024Modeling Unified Semantic Discourse Structure for High-quality Headline Generationcited by 0
  6. 2025Policy Learning with a Natural Language Action Space: A Causal Approachcited by 0
  7. 2026RH-RAG: Trustworthy Long-Form Generation for Privacy-Constrained Settingscited by 0