# QontSum: On Contrasting Salient Content for Query-focused Summarization

arXiv: https://arxiv.org/abs/2307.07586 · HTML: https://mlsanity.com/paper/2307.07586/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 57 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization](https://mlsanity.com/paper/2109.02492/) (2021)
- [SQuALITY: Building a Long-Document Summarization Dataset the Hard Way](https://mlsanity.com/paper/2205.11465/) (2022)
- [Exploring Neural Models for Query-Focused Summarization](https://mlsanity.com/paper/2112.07637/) (2021)
- [Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models](https://mlsanity.com/paper/1801.07704/) (2018)
- [Generating Query Focused Summaries from Query-Free Resources](https://mlsanity.com/paper/2012.14774/) (2020)
- [Domain Adaptation with Pre-trained Transformers for Query Focused Abstractive Text Summarization](https://mlsanity.com/paper/2112.11670/) (2021)
- [Adapting Pretrained Text-to-Text Models for Long Text Sequences](https://mlsanity.com/paper/2209.10052/) (2022)
- [Data Augmentation for Abstractive Query-Focused Multi-Document Summarization](https://mlsanity.com/paper/2103.01863/) (2021)
- [Long Context Question Answering via Supervised Contrastive Learning](https://mlsanity.com/paper/2112.08777/) (2021)
- [Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization](https://mlsanity.com/paper/2212.10449/) (2022)
- [Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance](https://mlsanity.com/paper/2105.12969/) (2021)
- [Abstractive Snippet Generation](https://mlsanity.com/paper/2002.10782/) (2020)
- [Decision-Focused Summarization](https://mlsanity.com/paper/2109.06896/) (2021)
- [DoSSIER@COLIEE 2021: Leveraging dense retrieval and summarization-based re-ranking for case law retrieval](https://mlsanity.com/paper/2108.03937/) (2021)
- [Towards Abstractive Grounded Summarization of Podcast Transcripts](https://mlsanity.com/paper/2203.11425/) (2022)
- [Few-shot Query-Focused Summarization with Prefix-Merging](https://mlsanity.com/paper/2211.16164/) (2022)

## Built on by (most cited first, 4 of 4)

- [Learning to Rank Salient Content for Query-focused Summarization](https://mlsanity.com/paper/2411.00324/) (2024)
- [IDEAL: Leveraging Infinite and Dynamic Characterizations of Large Language Models for Query-focused Summarization](https://mlsanity.com/paper/2407.10486/) (2024)
- [Does This Summary Answer My Question? Modeling Query-Focused Summary Readers with Rational Speech Acts](https://mlsanity.com/paper/2411.06524/) (2024)
- [Retrieved-Span Training for Efficient Query-Focused Meeting Summarization on QMSum](https://mlsanity.com/paper/2609.25028/) (2026)

## Abstract (arXiv)

Query-focused summarization (QFS) is a challenging task in natural language processing that generates summaries to address specific queries. The broader field of Generative Information Retrieval (Gen-IR) aims to revolutionize information extraction from vast document corpora through generative approaches, encompassing Generative Document Retrieval (GDR) and Grounded Answer Retrieval (GAR). This paper highlights the role of QFS in Grounded Answer Generation (GAR), a key subdomain of Gen-IR that produces human-readable answers in direct correspondence with queries, grounded in relevant documents. In this study, we propose QontSum, a novel approach for QFS that leverages contrastive learning to help the model attend to the most relevant regions of the input document. We evaluate our approach on a couple of benchmark datasets for QFS and demonstrate that it either outperforms existing state-of-the-art or exhibits a comparable performance with considerably reduced computational cost through enhancements in the fine-tuning stage, rather than relying on large-scale pre-training experiments, which is the focus of current SOTA. Moreover, we conducted a human study and identified improvements in the relevance of generated summaries to the posed queries without compromising fluency. We further conduct an error analysis study to understand our model's limitations and propose avenues for future research.