mlsanityData to 7 Oct 2026

Enriching and Controlling Global Semantics for Text Summarization

arXiv 2109.10616 ↗First posted 22 Sep 2021cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Recently, Transformer-based models have been proven effective in the abstractive summarization task by creating fluent and informative summaries. Nevertheless, these models still suffer from the short-range dependency problem, causing them to produce summaries that miss the key points of document. In this paper, we attempt to address this issue by introducing a neural topic model empowered with normalizing flow to capture the global semantics of the document, which are then integrated into the summarization model. In addition, to avoid the overwhelming effect of global semantics on contextualized representation, we introduce a mechanism to control the amount of global semantics supplied to the text generation module. Our method outperforms state-of-the-art summarization models on five common text summarization datasets, namely CNN/DailyMail, XSum, Reddit TIFU, arXiv, and PubMed.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 22 Sep 2021.

Builds on

34 of 43 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2015Variational Inference with Normalizing Flowscited by 2,161
  5. 2016Variational Inference: A Review for Statisticianscited by 1,466
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  9. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  10. 2015Teaching Machines to Read and Comprehendcited by 1,039
  11. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  12. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
22 more
  1. 2019Text Summarization with Pretrained Encoderscited by 553
  2. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  3. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  4. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
  5. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  6. 2020Extractive Summarization as Text Matchingcited by 189
  7. 2018Bottom-Up Abstractive Summarizationcited by 177
  8. 2019Fine-tune BERT for Extractive Summarizationcited by 122
  9. 2019HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarizationcited by 120
  10. 2018Abstractive Summarization of Reddit Posts with Multi-level Memory Networkscited by 102
  11. 2019On Extractive and Abstractive Neural Document Summarization with Transformer Language Modelscited by 83
  12. 2017Discovering Discrete Latent Topics with Neural Variational Inferencecited by 79
  13. 2020Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarizationcited by 72
  14. 2019Topic-aware Pointer-Generator Networks for Summarizing Spoken Conversationscited by 41
  15. 2018Coherence-Aware Neural Topic Modelingcited by 26
  16. 2018Topic Memory Networks for Short Text Classificationcited by 25
  17. 2018A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarizationcited by 21
  18. 2020Systematically Exploring Redundancy Reduction in Summarizing Long Documentscited by 21
  19. 2019Capturing Greater Context for Question Generationcited by 19
  20. 2020Stepwise Extractive Summarization and Planning with Structured Transformerscited by 16
  21. 2019Topic Augmented Generator for Abstractive Summarizationcited by 6
  22. 2020Incorporating Effective Global Information via Adaptive Gate Attention for Text Classificationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 4; 2024: 6; 2025: 2; 2026: 1.

20 papers.

Sort:

By area:

  1. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  2. 2021Contrastive Learning for Neural Topic Modelcited by 48
  3. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  4. 2021Improving Neural Cross-Lingual Summarization via Employing Optimal Transport Distance for Knowledge Distillationcited by 22
  5. 2024A Survey on Neural Topic Models: Methods, Applications, and Challengescited by 18
  6. 2022MVP: Multi-task Supervised Pre-training for Natural Language Generationcited by 16
  7. 2022Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictionscited by 13
  8. 2024Topic Modeling as Multi-Objective Contrastive Optimizationcited by 12
  9. 2022GRETEL: Graph Contrastive Topic Enhanced Language Model for Long Document Extractive Summarizationcited by 10
  10. 2024Encoding and Controlling Global Semantics for Long-form Video Question Answeringcited by 8
  11. 2023Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Predictioncited by 7
  12. 2024Curriculum Demonstration Selection for In-Context Learningcited by 5
8 more
  1. 2025CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Basecited by 4
  2. 2022Differentiable N-gram Objective on Abstractive Summarizationcited by 2
  3. 2023Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarizationcited by 1
  4. 2024Topic-aware Causal Intervention for Counterfactual Detectioncited by 1
  5. 2023Curriculum-Guided Abstractive Summarizationcited by 0
  6. 2023Boosting Summarization with Normalizing Flows and Aggressive Trainingcited by 0
  7. 2025Continual Neural Topic Modelcited by 0
  8. 2026Improving Topic Modeling by Distilling Soft Labels from Language Modelscited by 0