# SEMQA: Semi-Extractive Multi-Source Question Answering

arXiv: https://arxiv.org/abs/2311.04886 · HTML: https://mlsanity.com/paper/2311.04886/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 62 references that are in the map.
- First posted on arXiv 8 Nov 2023.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Enabling Large Language Models to Generate Text with Citations](https://mlsanity.com/paper/2305.14627/) (2023)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Measuring Attribution in Natural Language Generation Models](https://mlsanity.com/paper/2112.12870/) (2021)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://mlsanity.com/paper/2103.08541/) (2021)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [Scaling Up Models and Data with textttt5x and textttseqio](https://mlsanity.com/paper/2203.17189/) (2022)
- [Decontextualization: Making Sentences Stand-Alone](https://mlsanity.com/paper/2102.05169/) (2021)
- [The Limitations of Stylometry for Detecting Machine-Generated Fake News](https://mlsanity.com/paper/1908.09805/) (2019)
- [On the Limitations of Reference-Free Evaluations of Generated Text](https://mlsanity.com/paper/2210.12563/) (2022)
- [QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs](https://mlsanity.com/paper/2205.12665/) (2022)
- [EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start](https://mlsanity.com/paper/2205.12209/) (2022)

## Built on by (most cited first, 19 of 19)

- [Attribute First, then Generate: Locally-attributable Grounded Text Generation](https://mlsanity.com/paper/2403.17104/) (2024)
- [Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering](https://mlsanity.com/paper/2405.17980/) (2024)
- [AmbigDocs: Reasoning across Documents on Different Entities under the Same Name](https://mlsanity.com/paper/2404.12447/) (2024)
- [PLD+: Accelerating LLM inference by leveraging Language Model Artifacts](https://mlsanity.com/paper/2412.01447/) (2024)
- [GINGER: Grounded Information Nugget-Based Generation of Responses](https://mlsanity.com/paper/2503.18174/) (2025)
- [TokenShapley: Token Level Context Attribution with Shapley Value](https://mlsanity.com/paper/2507.05261/) (2025)
- [LAQuer: Localized Attribution Queries in Content-grounded Generation](https://mlsanity.com/paper/2506.01187/) (2025)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [ST-Raptor: LLM-Powered Semi-Structured Table Question Answering](https://mlsanity.com/paper/2508.18190/) (2025)
- [CoCoA: Confidence and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models](https://mlsanity.com/paper/2508.17670/) (2025)
- [The Extractive-Abstractive Spectrum: Uncovering Verifiability Trade-offs in LLM Generations](https://mlsanity.com/paper/2411.17375/) (2024)
- [Attribution, Citation, and Quotation: A Survey of Evidence-based Text Generation with Large Language Models](https://mlsanity.com/paper/2508.15396/) (2025)
- [Attention with Dependency Parsing Augmentation for Fine-Grained Attribution](https://mlsanity.com/paper/2412.11404/) (2024)
- [Efficient Real-time Refinement of Language Model Text Generation](https://mlsanity.com/paper/2501.07824/) (2025)
- [DBRouting: Routing End User Queries to Databases for Answerability](https://mlsanity.com/paper/2501.16220/) (2025)
- [Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models](https://mlsanity.com/paper/2506.17585/) (2025)
- [Cross-Document Event-Keyed Summarization](https://mlsanity.com/paper/2410.14795/) (2024)
- [DebugLM: Learning Traceable Training Data Provenance for LLMs](https://mlsanity.com/paper/2603.17884/) (2026)
- [Guaranteeing Faithful Evidence Extraction in Speculative Retrieval-Augmented Generation](https://mlsanity.com/paper/2609.10046/) (2026)

## Abstract (arXiv)

Recently proposed long-form question answering (QA) systems, supported by large language models (LLMs), have shown promising capabilities. Yet, attributing and verifying their generated abstractive answers can be difficult, and automatically evaluating their accuracy remains an ongoing challenge. In this work, we introduce a new QA task for answering multi-answer questions by summarizing multiple diverse sources in a semi-extractive fashion. Specifically, Semi-extractive Multi-source QA (SEMQA) requires models to output a comprehensive answer, while mixing factual quoted spans -- copied verbatim from given input sources -- and non-factual free-text connectors that glue these spans together into a single cohesive passage. This setting bridges the gap between the outputs of well-grounded but constrained extractive QA systems and more fluent but harder to attribute fully abstractive answers. Particularly, it enables a new mode for language models that leverages their advanced language generation capabilities, while also producing fine in-line attributions by-design that are easy to verify, interpret, and evaluate. To study this task, we create the first dataset of this kind, QuoteSum, with human-written semi-extractive answers to natural and generated questions, and define text-based evaluation metrics. Experimenting with several LLMs in various settings, we find this task to be surprisingly challenging, demonstrating the importance of QuoteSum for developing and studying such consolidation capabilities.