# Beyond Relevant Documents: A Knowledge-Intensive Approach for Query-Focused Summarization using Large Language Models

arXiv: https://arxiv.org/abs/2408.10357 · HTML: https://mlsanity.com/paper/2408.10357/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 63 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Overview of the TREC 2020 deep learning track](https://mlsanity.com/paper/2102.07662/) (2021)
- [AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarization](https://mlsanity.com/paper/2010.12694/) (2020)
- [Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval](https://mlsanity.com/paper/2212.10526/) (2022)
- [Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics](https://mlsanity.com/paper/2406.15264/) (2024)
- [Query-based Video Summarization with Pseudo Label Supervision](https://mlsanity.com/paper/2307.01945/) (2023)
- [A Novel Evaluation Framework for Image2Text Generation](https://mlsanity.com/paper/2408.01723/) (2024)
- [Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions](https://mlsanity.com/paper/2304.03147/) (2023)
- [Tackling Query-Focused Summarization as A Knowledge-Intensive Task: A Pilot Study](https://mlsanity.com/paper/2112.07536/) (2021)
- [ODSum: New Benchmarks for Open Domain Multi-Document Summarization](https://mlsanity.com/paper/2309.08960/) (2023)

## Built on by (most cited first, 4 of 4)

- [EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.12559/) (2024)
- [In-depth Research Impact Summarization through Fine-Grained Temporal Citation Analysis](https://mlsanity.com/paper/2505.14838/) (2025)
- [Recursive Abstractive Processing for Retrieval in Dynamic Datasets](https://mlsanity.com/paper/2410.01736/) (2024)
- [A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speech](https://mlsanity.com/paper/2604.06327/) (2026)

## Abstract (arXiv)

Query-focused summarization (QFS) is a fundamental task in natural language processing with broad applications, including search engines and report generation. However, traditional approaches assume the availability of relevant documents, which may not always hold in practical scenarios, especially in highly specialized topics. To address this limitation, we propose a novel knowledge-intensive approach that reframes QFS as a knowledge-intensive task setup. This approach comprises two main components: a retrieval module and a summarization controller. The retrieval module efficiently retrieves potentially relevant documents from a large-scale knowledge corpus based on the given textual query, eliminating the dependence on pre-existing document sets. The summarization controller seamlessly integrates a powerful large language model (LLM)-based summarizer with a carefully tailored prompt, ensuring the generated summary is comprehensive and relevant to the query. To assess the effectiveness of our approach, we create a new dataset, along with human-annotated relevance labels, to facilitate comprehensive evaluation covering both retrieval and summarization performance. Extensive experiments demonstrate the superior performance of our approach, particularly its ability to generate accurate summaries without relying on the availability of relevant documents initially. This underscores our method's versatility and practical applicability across diverse query scenarios.