# The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generations

arXiv: https://arxiv.org/abs/2411.17375 · HTML: https://mlsanity.com/paper/2411.17375/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Builds on (most cited first, 19 of 19)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Teaching language models to support answers with verified quotes](https://mlsanity.com/paper/2203.11147/) (2022)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?](https://mlsanity.com/paper/2308.10168/) (2023)
- [Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models](https://mlsanity.com/paper/2212.08037/) (2022)
- [ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reports](https://mlsanity.com/paper/2212.14882/) (2022)
- [Attribute First, then Generate: Locally-attributable Grounded Text Generation](https://mlsanity.com/paper/2403.17104/) (2024)
- [PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptions](https://mlsanity.com/paper/2305.14908/) (2023)
- [AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation](https://mlsanity.com/paper/2402.14978/) (2024)
- [How Susceptible are LLMs to Influence in Prompts?](https://mlsanity.com/paper/2408.11865/) (2024)
- [Intelligent Clinical Documentation: Harnessing Generative AI for Patient-Centric Clinical Note Generation](https://mlsanity.com/paper/2405.18346/) (2024)
- [SEMQA: Semi-Extractive Multi-Source Question Answering](https://mlsanity.com/paper/2311.04886/) (2023)
- [Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data](https://mlsanity.com/paper/2404.03862/) (2024)
- [A GEN AI Framework for Medical Note Generation](https://mlsanity.com/paper/2410.01841/) (2024)

## Built on by (most cited first, 8 of 8)

- [DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis](https://mlsanity.com/paper/2508.20033/) (2025)
- [Unstructured Evidence Attribution for Long Context Query Focused Summarization](https://mlsanity.com/paper/2502.14409/) (2025)
- [Transparentize the Internal and External Knowledge Utilization in LLMs with Trustworthy Citation](https://mlsanity.com/paper/2504.14856/) (2025)
- [Peerispect: Claim Verification in Scientific Peer Reviews](https://mlsanity.com/paper/2604.17667/) (2026)
- [Comparison of Unsupervised Metrics for Evaluating Judicial Decision Extraction](https://mlsanity.com/paper/2510.01792/) (2025)
- [AI Assistance for Human Review of Default Judgments](https://mlsanity.com/paper/2607.01256/) (2026)
- [Guaranteeing Faithful Evidence Extraction in Speculative Retrieval-Augmented Generation](https://mlsanity.com/paper/2609.10046/) (2026)
- [The Attribution-Compression Frontier in Retrieval-Augmented Generation](https://mlsanity.com/paper/2609.14245/) (2026)

## Abstract (arXiv)

Across all fields of academic study, experts cite their sources when sharing information. While large language models (LLMs) excel at synthesizing information, they do not provide reliable citation to sources, making it difficult to trace and verify the origins of the information they present. In contrast, search engines make sources readily accessible to users and place the burden of synthesizing information on the user. Through a survey, we find that users prefer search engines over LLMs for high-stakes queries, where concerns regarding information provenance outweigh the perceived utility of LLM responses. To examine the interplay between verifiability and utility of information-sharing tools, we introduce the extractive-abstractive spectrum, in which search engines and LLMs are extreme endpoints encapsulating multiple unexplored intermediate operating points. Search engines are extractive because they respond to queries with snippets of sources with links (citations) to the original webpages. LLMs are abstractive because they address queries with answers that synthesize and logically transform relevant information from training and in-context sources without reliable citation. We define five operating points that span the extractive-abstractive spectrum and conduct human evaluations on seven systems across four diverse query distributions that reflect real-world QA settings: web search, language simplification, multi-step reasoning, and medical advice. As outputs become more abstractive, we find that perceived utility improves by as much as 200%, while the proportion of properly cited sentences decreases by as much as 50% and users take up to 3 times as long to verify cited information. Our findings recommend distinct operating points for domain-specific LLM systems and our failure analysis informs approaches to high-utility LLM systems that empower users to verify information.