mlsanityData to 14 Oct 2026

Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization

arXiv 2205.14521 ↗First posted 28 May 2022cs.CL cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Text summarization aims to generate a short summary for an input text. In this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS) approach, which does not require parallel data for training. Our NAUS first performs edit-based search towards a heuristically defined score, and generates a summary as pseudo-groundtruth. Then, we train an encoder-only non-autoregressive Transformer based on the search result. We also propose a dynamic programming approach for length-control decoding, which is important for the summarization task. Experiments on two datasets show that NAUS achieves state-of-the-art performance for unsupervised summarization, yet largely improving inference efficiency. Further, our algorithm is able to perform explicit length-transfer summary generation.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 28 May 2022.

Builds on

33 of 55 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  3. 2017Non-Autoregressive Neural Machine Translationcited by 463
  4. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 396
  5. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 195
  6. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  7. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 157
  8. 2020If beam search is the answer, what was the question?cited by 116
  9. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  10. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  11. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  12. 2020Imputer: Sequence Modelling via Imputation and Dynamic Programmingcited by 66
21 more
  1. 2018Learning Neural Templates for Text Generationcited by 61
  2. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  3. 2019BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principlecited by 39
  4. 2019Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translationcited by 39
  5. 2019Unsupervised Paraphrasing by Simulated Annealingcited by 37
  6. 2020Unsupervised Text Generation by Learning from Searchcited by 36
  7. 2021Non-Autoregressive Text Generation with Pre-trained Language Modelscited by 36
  8. 2020Iterative Edit-Based Unsupervised Sentence Simplificationcited by 33
  9. 2020BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretrainingcited by 33
  10. 2020TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoisingcited by 30
  11. 2016Language as a Latent Variable: Discrete Generative Models for Sentence Compressioncited by 28
  12. 2019SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compressioncited by 27
  13. 2021RefSum: Refactoring Neural Summarizationcited by 25
  14. 2020Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extractioncited by 23
  15. 2019Simple Unsupervised Summarization by Contextual Matchingcited by 18
  16. 2018Unsupervised Sentence Compression using Denoising Auto-Encoderscited by 15
  17. 2021A New Approach to Overgenerating and Scoring Abstractive Summariescited by 13
  18. 2020Offline Reinforcement Learning from Human Feedback in Real-World Sequence-to-Sequence Taskscited by 12
  19. 2018Learning to Encode Text as Human-Readable Summaries using Generative Adversarial Networkscited by 11
  20. 2021Search and Learn: Improving Semantic Coverage for Data-to-Text Generationcited by 8
  21. 2021Simulated annealing for optimization of graphs and sequencescited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 8; 2024: 1; 2025: 1.

12 papers.

  1. 2023SLOG: A Structural Generalization Benchmark for Semantic Parsingcited by 22
  2. 2023Prompt-Based Editing for Text Style Transfercited by 16
  3. 2022Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillationcited by 14
  4. 2022Generating Multiple-Length Summaries via Reinforcement Learning for Unsupervised Sentence Summarizationcited by 5
  5. 2023The Emergence of Chunking Structures with Hierarchical RNNcited by 2
  6. 2023RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformercited by 1
  7. 2025Multilingual Non-Autoregressive Machine Translation without Knowledge Distillationcited by 1
  8. 2023Evaluating the Efficacy of Length-Controllable Machine Translationcited by 0
  9. 2023Recent Trends in Unsupervised Summarizationcited by 0
  10. 2023KEST: Kernel Distance Based Efficient Self-Training for Improving Controllable Text Generationcited by 0
  11. 2023From Lengthy to Lucid: A Systematic Literature Review on NLP Techniques for Taming Long Sentencescited by 0
  12. 2024Query-Guided Self-Supervised Summarization of Nursing Notescited by 0