mlsanityData to 7 Oct 2026

Knowledge Transfer from Answer Ranking to Answer Generation

arXiv 2210.12865 ↗First posted 23 Oct 2022cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent studies show that Question Answering (QA) based on Answer Sentence Selection (AS2) can be improved by generating an improved answer from the top-k ranked answer sentences (termed GenQA). This allows for synthesizing the information from multiple candidates into a concise, natural-sounding answer. However, creating large-scale supervised training data for GenQA models is very challenging. In this paper, we propose to train a GenQA model by transferring knowledge from a trained AS2 model, to overcome the aforementioned issue. First, we use an AS2 model to produce a ranking over answer candidates for a set of questions. Then, we use the top ranked candidate as the generation target, and the next k top ranked candidates as context for training a GenQA model. We also propose to use the AS2 model prediction scores for loss weighting and score-conditioned input/output shaping, to aid the knowledge transfer. Our evaluation on three public and one large industrial datasets demonstrates the superiority of our approach over the AS2 baseline, and GenQA trained using supervised data.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 32 of its 53 references that are in the map. First posted on arXiv 23 Oct 2022.

Builds on

32 of 53 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
20 more
  1. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  2. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  3. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  4. 2020Selective Question Answering under Domain Shiftcited by 228
  5. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 124
  6. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  7. 2015Multi-task Sequence to Sequence Learningcited by 94
  8. 2019TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selectioncited by 74
  9. 2021Evidentiality-guided Generation for Knowledge-Intensive NLP Taskscited by 40
  10. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  11. 2019Joint Learning of Answer Selection and Answer Summary Generation in Community Question Answeringcited by 25
  12. 2016A Compare-Aggregate Model for Matching Text Sequencescited by 23
  13. 2019A Compare-Aggregate Model with Latent Clustering for Answer Selectioncited by 19
  14. 2021Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answeringcited by 19
  15. 2021Attention-guided Generative Models for Extractive Question Answeringcited by 14
  16. 2021Answer Generation for Retrieval-based Question Answering Systemscited by 13
  17. 2021Joint Models for Answer Verification in Question Answering Systemscited by 13
  18. 2022Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selectioncited by 9
  19. 2022Paragraph-based Transformer Pre-training for Multi-Sentence Inferencecited by 7
  20. 2022Double Retrieval and Ranking for Accurate Question Answeringcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 2.

6 papers.

  1. 2023Learning Answer Generation using Supervision from Automatic Question Answering Evaluatorscited by 3
  2. 2024Measuring Retrieval Complexity in Question Answering Systemscited by 3
  3. 2023Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languagescited by 2
  4. 2023Context-Aware Transformer Pre-Training for Answer Sentence Selectioncited by 1
  5. 2023SQUARE: Automatic Question Answering Evaluation using Multiple Positive and Negative Referencescited by 1
  6. 2024Datasets for Multilingual Answer Sentence Selectioncited by 0