# Diversity driven Attention Model for Query-based Abstractive Summarization

arXiv: https://arxiv.org/abs/1704.08300 · HTML: https://mlsanity.com/paper/1704.08300/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2017.

## Built on by (most cited first, 30 of 42)

- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [OpenCQA: Open-ended Question Answering with Charts](https://mlsanity.com/paper/2210.06628/) (2022)
- [AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization](https://mlsanity.com/paper/2103.11332/) (2021)
- [SQuALITY: Building a Long-Document Summarization Dataset the Hard Way](https://mlsanity.com/paper/2205.11465/) (2022)
- [Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2309.17050/) (2023)
- [An Exploratory Study on Long Dialogue Summarization: What Works and What's Next](https://mlsanity.com/paper/2109.04609/) (2021)
- [Domain Adaptation with Pre-trained Transformers for Query Focused Abstractive Text Summarization](https://mlsanity.com/paper/2112.11670/) (2021)
- [Controllable Summarization with Constrained Markov Decision Process](https://mlsanity.com/paper/2108.03405/) (2021)
- [Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance](https://mlsanity.com/paper/2105.12969/) (2021)
- [Context-aware Decoding Reduces Hallucination in Query-focused Summarization](https://mlsanity.com/paper/2312.14335/) (2023)
- [Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension](https://mlsanity.com/paper/2109.06853/) (2021)
- [Monotonic Paraphrasing Improves Generalization of Language Model Prompting](https://mlsanity.com/paper/2403.16038/) (2024)
- [LMGQS: A Large-scale Dataset for Query-focused Summarization](https://mlsanity.com/paper/2305.13086/) (2023)
- [An Overview on Generative AI at Scale with Edge-Cloud Computing](https://mlsanity.com/paper/2306.17170/) (2023)
- [Text Summarization with Latent Queries](https://mlsanity.com/paper/2106.00104/) (2021)
- [A Lightweight Constrained Generation Alternative for Query-focused Summarization](https://mlsanity.com/paper/2304.11721/) (2023)
- [CQSumDP: A ChatGPT-Annotated Resource for Query-Focused Abstractive Summarization Based on Debatepedia](https://mlsanity.com/paper/2305.06147/) (2023)
- [Efficient Diversity-based Experience Replay for Deep Reinforcement Learning](https://mlsanity.com/paper/2410.20487/) (2024)
- [SummerTime: Text Summarization Toolkit for Non-experts](https://mlsanity.com/paper/2108.12738/) (2021)
- [Why Do You Feel This Way? Summarizing Triggers of Emotions in Social Media Posts](https://mlsanity.com/paper/2210.12531/) (2022)
- [Few-shot Query-Focused Summarization with Prefix-Merging](https://mlsanity.com/paper/2211.16164/) (2022)
- [Tackling Query-Focused Summarization as A Knowledge-Intensive Task: A Pilot Study](https://mlsanity.com/paper/2112.07536/) (2021)
- [Deep Dynamic Neural Network to trade-off between Accuracy and Diversity in a News Recommender System](https://mlsanity.com/paper/2103.08458/) (2021)
- [Improving Online Forums Summarization via Hierarchical Unified Deep Neural Network](https://mlsanity.com/paper/2103.13587/) (2021)
- [PREME: Preference-based Meeting Exploration through an Interactive Questionnaire](https://mlsanity.com/paper/2205.02370/) (2022)
- [Survey of Query-based Text Summarization](https://mlsanity.com/paper/2211.11548/) (2022)
- [Saved You A Click: Automatically Answering Clickbait Titles](https://mlsanity.com/paper/2212.08196/) (2022)
- [Surveying the Landscape of Text Summarization with Deep Learning: A Comprehensive Review](https://mlsanity.com/paper/2310.09411/) (2023)
- [Instructive Dialogue Summarization with Query Aggregations](https://mlsanity.com/paper/2310.10981/) (2023)

## Abstract (arXiv)

Abstractive summarization aims to generate a shorter version of the document covering all the salient points in a compact and coherent fashion. On the other hand, query-based summarization highlights those points that are relevant in the context of a given query. The encode-attend-decode paradigm has achieved notable success in machine translation, extractive summarization, dialog systems, etc. But it suffers from the drawback of generation of repeated phrases. In this work we propose a model for the query-based summarization task based on the encode-attend-decode paradigm with two key additions (i) a query attention model (in addition to document attention model) which learns to focus on different portions of the query at different time steps (instead of using a static representation for the query) and (ii) a new diversity based attention model which aims to alleviate the problem of repeating phrases in the summary. In order to enable the testing of this model we introduce a new query-based summarization dataset building on debatepedia. Our experiments show that with these two additions the proposed model clearly outperforms vanilla encode-attend-decode models with a gain of 28% (absolute) in ROUGE-L scores.