# Neural Abstractive Text Summarization with Sequence-to-Sequence Models

arXiv: https://arxiv.org/abs/1812.02303 · HTML: https://mlsanity.com/paper/1812.02303/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2018.

## Built on by (most cited first, 30 of 39)

- [Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization](https://mlsanity.com/paper/2309.07430/) (2023)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [Time Waits for No One! Analysis and Challenges of Temporal Misalignment](https://mlsanity.com/paper/2111.07408/) (2021)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [Loss Functions and Metrics in Deep Learning](https://mlsanity.com/paper/2307.02694/) (2023)
- [A Review of Text Style Transfer using Deep Learning](https://mlsanity.com/paper/2109.15144/) (2021)
- [An Extractive-and-Abstractive Framework for Source Code Summarization](https://mlsanity.com/paper/2206.07245/) (2022)
- [Abstractive Text Summarization: State of the Art, Challenges, and Improvements](https://mlsanity.com/paper/2409.02413/) (2024)
- [Cheetah: Natural Language Generation for 517 African Languages](https://mlsanity.com/paper/2401.01053/) (2024)
- [Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation](https://mlsanity.com/paper/2106.06125/) (2021)
- [Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder](https://mlsanity.com/paper/2107.00189/) (2021)
- [Automated News Summarization Using Transformers](https://mlsanity.com/paper/2108.01064/) (2021)
- [Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restoration](https://mlsanity.com/paper/2205.10195/) (2022)
- [Who are you referring to? Coreference resolution in image narrations](https://mlsanity.com/paper/2211.14563/) (2022)
- [An Analysis of Abstractive Text Summarization Using Pre-trained Models](https://mlsanity.com/paper/2303.12796/) (2023)
- [An Evaluation of Large Language Models on Text Summarization Tasks Using Prompt Engineering Techniques](https://mlsanity.com/paper/2507.05123/) (2025)
- [Semi-supervised multimodal coreference resolution in image narrations](https://mlsanity.com/paper/2310.13619/) (2023)
- [Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches and Future Directions](https://mlsanity.com/paper/2210.09894/) (2022)
- [Think Carefully and Check Again! Meta-Generation Unlocking LLMs for Low-Resource Cross-Lingual Summarization](https://mlsanity.com/paper/2410.20021/) (2024)
- [Who speaks like a style of Vitamin: Towards Syntax-Aware DialogueSummarization using Multi-task Learning](https://mlsanity.com/paper/2109.14199/) (2021)
- [Abstractive Text Summarization using Attentive GRU based Encoder-Decoder](https://mlsanity.com/paper/2302.13117/) (2023)
- [Domain-specific Guided Summarization for Mental Health Posts](https://mlsanity.com/paper/2411.01485/) (2024)
- [Neural Abstractive Unsupervised Summarization of Online News Discussions](https://mlsanity.com/paper/2106.03953/) (2021)
- [Selective Token Generation for Few-shot Natural Language Generation](https://mlsanity.com/paper/2209.08206/) (2022)
- [Semantic Similarity Measure of Natural Language Text through Machine Learning and a Keyword-Aware Cross-Encoder-Ranking Summarizer -- A Case Study Using UCGIS GIS&T Body of Knowledge](https://mlsanity.com/paper/2305.09877/) (2023)
- [More is More: Addition Bias in Large Language Models](https://mlsanity.com/paper/2409.02569/) (2024)
- [asLLR: LLM based Leads Ranking in Auto Sales](https://mlsanity.com/paper/2510.21713/) (2025)
- [Using Natural Language Processing to Understand Reasons and Motivators Behind Customer Calls in Financial Domain](https://mlsanity.com/paper/2110.09094/) (2021)
- [Domain Specific Fine-tuning of Denoising Sequence-to-Sequence Models for Natural Language Summarization](https://mlsanity.com/paper/2204.09716/) (2022)

## Abstract (arXiv)

In the past few years, neural abstractive text summarization with sequence-to-sequence (seq2seq) models have gained a lot of popularity. Many interesting techniques have been proposed to improve seq2seq models, making them capable of handling different challenges, such as saliency, fluency and human readability, and generate high-quality summaries. Generally speaking, most of these techniques differ in one of these three categories: network structure, parameter inference, and decoding/generation. There are also other concerns, such as efficiency and parallelism for training a model. In this paper, we provide a comprehensive literature survey on different seq2seq models for abstractive text summarization from the viewpoint of network structures, training strategies, and summary generation algorithms. Several models were first proposed for language modeling and generation tasks, such as machine translation, and later applied to abstractive text summarization. Hence, we also provide a brief review of these models. As part of this survey, we also develop an open source library, namely, Neural Abstractive Text Summarizer (NATS) toolkit, for the abstractive text summarization. An extensive set of experiments have been conducted on the widely used CNN/Daily Mail dataset to examine the effectiveness of several different neural network components. Finally, we benchmark two models implemented in NATS on the two recently released datasets, namely, Newsroom and Bytecup.