# Answer Generation for Retrieval-based Question Answering Systems

arXiv: https://arxiv.org/abs/2106.00955 · HTML: https://mlsanity.com/paper/2106.00955/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models](https://mlsanity.com/paper/2101.05667/) (2021)
- [TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection](https://mlsanity.com/paper/1911.04118/) (2019)
- [Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2010.03546/) (2020)
- [The price of debiasing automatic metrics in natural language evaluation](https://mlsanity.com/paper/1807.02202/) (2018)
- [Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2001.11458/) (2020)
- [Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answering](https://mlsanity.com/paper/1911.09801/) (2019)
- [The Cascade Transformer: an Application for Efficient Answer Sentence Selection](https://mlsanity.com/paper/2005.02534/) (2020)
- [Modeling Context in Answer Sentence Selection Systems on a Latency Budget](https://mlsanity.com/paper/2101.12093/) (2021)

## Built on by (most cited first, 13 of 13)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [DocFinQA: A Long-Context Financial Reasoning Dataset](https://mlsanity.com/paper/2401.06915/) (2024)
- [Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestion](https://mlsanity.com/paper/2311.06318/) (2023)
- [Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answering](https://mlsanity.com/paper/2306.16713/) (2023)
- [Knowledge Transfer from Answer Ranking to Answer Generation](https://mlsanity.com/paper/2210.12865/) (2022)
- [Learning Answer Generation using Supervision from Automatic Question Answering Evaluators](https://mlsanity.com/paper/2305.15344/) (2023)
- [Measuring Retrieval Complexity in Question Answering Systems](https://mlsanity.com/paper/2406.03592/) (2024)
- [Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages](https://mlsanity.com/paper/2305.16302/) (2023)
- [FastFiD: Improve Inference Efficiency of Open Domain Question Answering via Sentence Selection](https://mlsanity.com/paper/2408.06333/) (2024)
- [SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative References](https://mlsanity.com/paper/2309.12250/) (2023)
- [The illusion of a perfect metric: Why evaluating AI's words is harder than it looks](https://mlsanity.com/paper/2508.13816/) (2025)
- [Question-Context Alignment and Answer-Context Dependencies for Effective Answer Sentence Selection](https://mlsanity.com/paper/2306.02196/) (2023)
- [BEST-KAG: Enhancing Question Answering of Building Engineering Standards with Multimodal Knowledge Graph Modeling and Large Language Model](https://mlsanity.com/paper/2608.11244/) (2026)

## Abstract (arXiv)

Recent advancements in transformer-based models have greatly improved the ability of Question Answering (QA) systems to provide correct answers; in particular, answer sentence selection (AS2) models, core components of retrieval-based systems, have achieved impressive results. While generally effective, these models fail to provide a satisfying answer when all retrieved candidates are of poor quality, even if they contain correct information. In AS2, models are trained to select the best answer sentence among a set of candidates retrieved for a given question. In this work, we propose to generate answers from a set of AS2 top candidates. Rather than selecting the best candidate, we train a sequence to sequence transformer model to generate an answer from a candidate set. Our tests on three English AS2 datasets show improvement up to 32 absolute points in accuracy over the state of the art.