# Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation

arXiv: https://arxiv.org/abs/1805.11004 · HTML: https://mlsanity.com/paper/1805.11004/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 25 of 25)

- [Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors](https://mlsanity.com/paper/2205.12854/) (2022)
- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Zero-shot Fact Verification by Claim Generation](https://mlsanity.com/paper/2105.14682/) (2021)
- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand](https://mlsanity.com/paper/2112.04139/) (2021)
- [The Summary Loop: Learning to Write Abstractive Summaries Without Examples](https://mlsanity.com/paper/2105.05361/) (2021)
- [Spurious Correlations in Reference-Free Evaluation of Text Generation](https://mlsanity.com/paper/2204.09890/) (2022)
- [Legal Judgment Prediction with Multi-Stage CaseRepresentation Learning in the Real Court Setting](https://mlsanity.com/paper/2107.05192/) (2021)
- [Scalable Educational Question Generation with Pre-trained Language Models](https://mlsanity.com/paper/2305.07871/) (2023)
- [Data Augmentation for Abstractive Query-Focused Multi-Document Summarization](https://mlsanity.com/paper/2103.01863/) (2021)
- [Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training](https://mlsanity.com/paper/2111.01338/) (2021)
- [NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist](https://mlsanity.com/paper/2305.08566/) (2023)
- [A New Approach to Overgenerating and Scoring Abstractive Summaries](https://mlsanity.com/paper/2104.01726/) (2021)
- [MultiOpEd: A Corpus of Multi-Perspective News Editorials](https://mlsanity.com/paper/2106.02725/) (2021)
- [An Evaluation Framework for Legal Document Summarization](https://mlsanity.com/paper/2205.08478/) (2022)
- [GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation](https://mlsanity.com/paper/2207.14467/) (2022)
- [XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags](https://mlsanity.com/paper/2406.03776/) (2024)
- [Enhance Long Text Understanding via Distilled Gist Detector from Abstractive Summarization](https://mlsanity.com/paper/2110.04741/) (2021)
- [Exploring Multitask Learning for Low-Resource AbstractiveSummarization](https://mlsanity.com/paper/2109.08565/) (2021)
- [Multi-Perspective Abstractive Answer Summarization](https://mlsanity.com/paper/2104.08536/) (2021)
- [Universal Evasion Attacks on Summarization Scoring](https://mlsanity.com/paper/2210.14260/) (2022)
- [Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generation](https://mlsanity.com/paper/2301.02275/) (2023)
- [DeepHelp: Deep Learning for Shout Crisis Text Conversations](https://mlsanity.com/paper/2110.13244/) (2021)
- [Sample Efficient Multimodal Semantic Augmentation for Incremental Summarization](https://mlsanity.com/paper/2303.04361/) (2023)
- [How Vital is the Jurisprudential Relevance: Law Article Intervened Legal Case Retrieval and Matching](https://mlsanity.com/paper/2502.18292/) (2025)

## Abstract (arXiv)

An accurate abstractive summary of a document should contain all its salient information and should be logically entailed by the input document. We improve these important aspects of abstractive summarization via multi-task learning with the auxiliary tasks of question generation and entailment generation, where the former teaches the summarization model how to look for salient questioning-worthy details, and the latter teaches the model how to rewrite a summary which is a directed-logical subset of the input document. We also propose novel multi-task architectures with high-level (semantic) layer-specific sharing across multiple encoder and decoder layers of the three tasks, as well as soft-sharing mechanisms (and show performance ablations and analysis examples of each contribution). Overall, we achieve statistically significant improvements over the state-of-the-art on both the CNN/DailyMail and Gigaword datasets, as well as on the DUC-2002 transfer setup. We also present several quantitative and qualitative analysis studies of our model's learned saliency and entailment skills.