# Context-aware Decoding Reduces Hallucination in Query-focused Summarization

arXiv: https://arxiv.org/abs/2312.14335 · HTML: https://mlsanity.com/paper/2312.14335/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [Trusting Your Evidence: Hallucinate Less with Context-aware Decoding](https://mlsanity.com/paper/2305.14739/) (2023)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints](https://mlsanity.com/paper/2010.12884/) (2020)
- [Contrastive Decoding Improves Reasoning in Large Language Models](https://mlsanity.com/paper/2309.09117/) (2023)
- [Mutual Information Alleviates Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2210.13210/) (2022)
- [FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge](https://mlsanity.com/paper/2305.08281/) (2023)
- [Diversity driven Attention Model for Query-based Abstractive Summarization](https://mlsanity.com/paper/1704.08300/) (2017)
- [Abstractive Snippet Generation](https://mlsanity.com/paper/2002.10782/) (2020)
- [A Lightweight Constrained Generation Alternative for Query-focused Summarization](https://mlsanity.com/paper/2304.11721/) (2023)
- [Counterfactual Editing for Search Result Explanation](https://mlsanity.com/paper/2301.10389/) (2023)

## Built on by (most cited first, 11 of 11)

- [MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation](https://mlsanity.com/paper/2501.00332/) (2024)
- [Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression](https://mlsanity.com/paper/2407.04965/) (2024)
- [TC-RAG:Turing-Complete RAG's Case study on Medical LLM Systems](https://mlsanity.com/paper/2408.09199/) (2024)
- [HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses](https://mlsanity.com/paper/2312.15883/) (2023)
- [RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of Transformers](https://mlsanity.com/paper/2403.18276/) (2024)
- [State Space Models are Strong Text Rerankers](https://mlsanity.com/paper/2412.14354/) (2024)
- [CSPLADE: Learned Sparse Retrieval with Causal Language Models](https://mlsanity.com/paper/2504.10816/) (2025)
- [Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models](https://mlsanity.com/paper/2410.03026/) (2024)
- [Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.13272/) (2025)
- [3DS: Medical Domain Adaptation of LLMs via Decomposed Difficulty-based Data Selection](https://mlsanity.com/paper/2410.10901/) (2024)
- [SFAD: Speculative Factuality-Aware Decoding](https://mlsanity.com/paper/2609.00796/) (2026)

## Abstract (arXiv)

Query-focused summarization (QFS) aims to provide a summary of a single document/multi documents that can satisfy the information needs of a given query. It is useful for various real-world applications, such as abstractive snippet generation or more recent retrieval augmented generation (RAG). A prototypical QFS pipeline consists of a retriever (sparse or dense retrieval) and a generator (usually a large language model). However, applying large language models (LLM) potentially leads to hallucinations, especially when the evidence contradicts the prior belief of LLMs. There has been growing interest in developing new decoding methods to improve generation quality and reduce hallucination. In this work, we conduct a large-scale reproducibility study on one recently proposed decoding method, -- ,Context-aware Decoding (CAD). In addition to replicating CAD's experiments on news summarization datasets, we include experiments on QFS datasets, and conduct more rigorous analysis on computational complexity and hyperparameter sensitivity. Experiments with eight different language models show that performance-wise, CAD improves QFS quality by (1) reducing factuality errors/hallucinations while (2) mostly retaining the match of lexical patterns, measured by ROUGE scores, while also at a cost of increased inference-time FLOPs and reduced decoding speed. The hrefhttps://github.com/zhichaoxu-shufe/context-aware-decoding-qfscode implementation based on Huggingface Library is made available