mlsanityData to 7 Oct 2026

Learning Answer Generation using Supervision from Automatic Question Answering Evaluators

arXiv 2305.15344 ↗First posted 24 May 2023cs.CL cs.LGcited by 3

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Recent studies show that sentence-level extractive QA, i.e., based on Answer Sentence Selection (AS2), is outperformed by Generation-based QA (GenQA) models, which generate answers using the top-k answer sentences ranked by AS2 models (a la retrieval-augmented generation style). In this paper, we propose a novel training paradigm for GenQA using supervision from automatic QA evaluation models (GAVA). Specifically, we propose three strategies to transfer knowledge from these QA evaluation models to a GenQA model: (i) augmenting training data with answers generated by the GenQA model and labelled by GAVA (either statically, before training, or (ii) dynamically, at every training epoch); and (iii) using the GAVA score for weighting the generator loss during the learning of the GenQA model. We evaluate our proposed methods on two academic and one industrial dataset, obtaining a significant improvement in answering accuracy over the previous state of the art.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 30 of its 41 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

30 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks16 Dec

Other references · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  9. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  10. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  11. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  12. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
17 more
  1. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  2. 2022On Decoding Strategies for Neural Text Generatorscited by 89
  3. 2019TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selectioncited by 74
  4. 2022Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluationcited by 66
  5. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  6. 2019Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answeringcited by 25
  7. 2021What's in a Name? Answer Equivalence For Open-Domain Question Answeringcited by 21
  8. 2018Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Taskcited by 20
  9. 2021Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answeringcited by 19
  10. 2021Attention-guided Generative Models for Extractive Question Answeringcited by 14
  11. 2021Answer Generation for Retrieval-based Question Answering Systemscited by 13
  12. 2021Joint Models for Answer Verification in Question Answering Systemscited by 13
  13. 2020AVA: an Automatic eValuation Approach to Question Answering Systemscited by 10
  14. 2022Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selectioncited by 9
  15. 2022Paragraph-based Transformer Pre-training for Multi-Sentence Inferencecited by 7
  16. 2021Cross-Lingual Open-Domain Question Answering with Answer Sentence Generationcited by 6
  17. 2022Knowledge Transfer from Answer Ranking to Answer Generationcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2023PACE: Improving Prompt with Actor-Critic Editing for Large Language Modelcited by 20
  2. 2023SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative Referencescited by 1
  3. 2024Datasets for Multilingual Answer Sentence Selectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Automatic Dataset Generation for Knowledge Intensive Question Answering Tasks
  2. 2025Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models
  3. 2024GenSco: Can Question Decomposition based Passage Alignment improve Question Answering?
  4. 2022Leveraging QA Datasets to Improve Generative Data Augmentation
  5. 2024GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answering
  6. 2024Generative Language Models with Retrieval Augmented Generation for Automated Short Answer Scoring
  7. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering
  8. 2024RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering