# A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss

arXiv: https://arxiv.org/abs/1805.06266 · HTML: https://mlsanity.com/paper/1805.06266/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2018.

## Built on by (most cited first, 30 of 47)

- [XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages](https://mlsanity.com/paper/2106.13822/) (2021)
- [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://mlsanity.com/paper/2205.12854/) (2022)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [Submodularity In Machine Learning and Artificial Intelligence](https://mlsanity.com/paper/2202.00132/) (2022)
- [Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand](https://mlsanity.com/paper/2112.04139/) (2021)
- [Towards Improving Faithfulness in Abstractive Summarization](https://mlsanity.com/paper/2210.01877/) (2022)
- [Spurious Correlations in Reference-Free Evaluation of Text Generation](https://mlsanity.com/paper/2204.09890/) (2022)
- [A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation](https://mlsanity.com/paper/2204.09867/) (2022)
- [Long-Span Summarization via Local Attention and Content Selection](https://mlsanity.com/paper/2105.03801/) (2021)
- [Topic-Aware Contrastive Learning for Abstractive Dialogue Summarization](https://mlsanity.com/paper/2109.04994/) (2021)
- [ACLSum: A New Dataset for Aspect-based Summarization of Scientific Publications](https://mlsanity.com/paper/2403.05303/) (2024)
- [Word Graph Guided Summarization for Radiology Findings](https://mlsanity.com/paper/2112.09925/) (2021)
- [An Extractive-and-Abstractive Framework for Source Code Summarization](https://mlsanity.com/paper/2206.07245/) (2022)
- [Abstractive Text Summarization: State of the Art, Challenges, and Improvements](https://mlsanity.com/paper/2409.02413/) (2024)
- [NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist](https://mlsanity.com/paper/2305.08566/) (2023)
- [Summarization Programs: Interpretable Abstractive Summarization with Neural Modular Trees](https://mlsanity.com/paper/2209.10492/) (2022)
- [Salience Allocation as Guidance for Abstractive Summarization](https://mlsanity.com/paper/2210.12330/) (2022)
- [Differentiable Multi-Agent Actor-Critic for Multi-Step Radiology Report Summarization](https://mlsanity.com/paper/2203.08257/) (2022)
- [Robust Natural Language Understanding with Residual Attention Debiasing](https://mlsanity.com/paper/2305.17627/) (2023)
- [Contextualized Rewriting for Text Summarization](https://mlsanity.com/paper/2102.00385/) (2021)
- [Towards Abstractive Grounded Summarization of Podcast Transcripts](https://mlsanity.com/paper/2203.11425/) (2022)
- [Concise Answers to Complex Questions: Summarization of Long-form Answers](https://mlsanity.com/paper/2305.19271/) (2023)
- [MHMS: Multimodal Hierarchical Multimedia Summarization](https://mlsanity.com/paper/2204.03734/) (2022)
- [Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment](https://mlsanity.com/paper/2210.04722/) (2022)
- [Follow the Timeline! Generating Abstractive and Extractive Timeline Summary in Chronological Order](https://mlsanity.com/paper/2301.00867/) (2023)
- [Exploring Multitask Learning for Low-Resource AbstractiveSummarization](https://mlsanity.com/paper/2109.08565/) (2021)
- [EASE: Extractive-Abstractive Summarization with Explanations](https://mlsanity.com/paper/2105.06982/) (2021)
- [Revision for Concision: A Constrained Paraphrase Generation Task](https://mlsanity.com/paper/2210.14257/) (2022)
- [Sparsity and Sentence Structure in Encoder-Decoder Attention of Summarization Systems](https://mlsanity.com/paper/2109.03888/) (2021)
- [Dialogue Inspectional Summarization with Factual Inconsistency Awareness](https://mlsanity.com/paper/2111.03284/) (2021)

## Abstract (arXiv)

We propose a unified model combining the strength of extractive and abstractive summarization. On the one hand, a simple extractive model can obtain sentence-level attention with high ROUGE scores but less readable. On the other hand, a more complicated abstractive model can obtain word-level dynamic attention to generate a more readable paragraph. In our model, sentence-level attention is used to modulate the word-level attention such that words in less attended sentences are less likely to be generated. Moreover, a novel inconsistency loss function is introduced to penalize the inconsistency between two levels of attentions. By end-to-end training our model with the inconsistency loss and original losses of extractive and abstractive models, we achieve state-of-the-art ROUGE scores while being the most informative and readable summarization on the CNN/Daily Mail dataset in a solid human evaluation.