# Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models

arXiv: https://arxiv.org/abs/2302.05578 · HTML: https://mlsanity.com/paper/2302.05578/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 11 Feb 2023.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 15 of 15)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models](https://mlsanity.com/paper/2205.11503/) (2022)
- [Evidence-based Factual Error Correction](https://mlsanity.com/paper/2012.15788/) (2020)
- [Decoupled Context Processing for Context Augmented Language Modeling](https://mlsanity.com/paper/2210.05758/) (2022)

## Built on by (most cited first, 13 of 13)

- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection](https://mlsanity.com/paper/2310.09044/) (2023)
- [Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning](https://mlsanity.com/paper/2404.08680/) (2024)
- [Elastic Weight Removal for Faithful and Abstractive Dialogue Generation](https://mlsanity.com/paper/2303.17574/) (2023)
- [REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy](https://mlsanity.com/paper/2406.07735/) (2024)
- [AutoHall: Automated Factuality Hallucination Dataset Generation for Large Language Models](https://mlsanity.com/paper/2310.00259/) (2023)
- [KL-Divergence Guided Temperature Sampling](https://mlsanity.com/paper/2306.01286/) (2023)
- [Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities](https://mlsanity.com/paper/2405.00711/) (2024)
- [Automatic Glossary of Clinical Terminology: a Large-Scale Dictionary of Biomedical Definitions Generated from Ontological Knowledge](https://mlsanity.com/paper/2306.00665/) (2023)
- [Characterizing Tradeoffs in Language Model Decoding with Informational Interpretations](https://mlsanity.com/paper/2311.10083/) (2023)

## Abstract (arXiv)

Despite recent progress, it has been difficult to prevent semantic hallucinations in generative Large Language Models. One common solution to this is augmenting LLMs with a retrieval system and making sure that the generated output is attributable to the retrieved information. Given this new added constraint, it is plausible to expect that the overall quality of the output will be affected, for example, in terms of fluency. Can scaling language models help? Here we examine the relationship between fluency and attribution in LLMs prompted with retrieved evidence in knowledge-heavy dialog settings. Our experiments were implemented with a set of auto-metrics that are aligned with human preferences. They were used to evaluate a large set of generations, produced under varying parameters of LLMs and supplied context. We show that larger models tend to do much better in both fluency and attribution, and that (naively) using top-k retrieval versus top-1 retrieval improves attribution but hurts fluency. We next propose a recipe that could allow smaller models to both close the gap with larger models and preserve the benefits of top-k retrieval while avoiding its drawbacks.