# Synthetic Question Value Estimation for Domain Adaptation of Question Answering

arXiv: https://arxiv.org/abs/2203.08926 · HTML: https://mlsanity.com/paper/2203.08926/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 47 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 30 of 37)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Training Deep Nets with Sublinear Memory Cost](https://mlsanity.com/paper/1604.06174/) (2016)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Towards Efficient Data Valuation Based on the Shapley Value](https://mlsanity.com/paper/1902.10275/) (2019)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://mlsanity.com/paper/1910.09753/) (2019)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [Data Valuation using Reinforcement Learning](https://mlsanity.com/paper/1909.11671/) (2019)
- [Training Question Answering Models From Synthetic Data](https://mlsanity.com/paper/2002.09599/) (2020)
- [Neural Question Generation from Text: A Preliminary Study](https://mlsanity.com/paper/1704.01792/) (2017)
- [End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems](https://mlsanity.com/paper/2010.06028/) (2020)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering](https://mlsanity.com/paper/1909.06356/) (2019)
- [Unsupervised Question Answering by Cloze Translation](https://mlsanity.com/paper/1906.04980/) (2019)
- [Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia](https://mlsanity.com/paper/1805.05942/) (2018)
- [Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus](https://mlsanity.com/paper/2002.00748/) (2020)
- [Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering](https://mlsanity.com/paper/2004.11892/) (2020)
- [Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs](https://mlsanity.com/paper/2005.13837/) (2020)
- [Neural Domain Adaptation for Biomedical Question Answering](https://mlsanity.com/paper/1706.03610/) (2017)
- [Supervised and Unsupervised Transfer Learning for Question Answering](https://mlsanity.com/paper/1711.05345/) (2017)
- [Simple and Effective Semi-Supervised Question Answering](https://mlsanity.com/paper/1804.00720/) (2018)
- [Adversarial Domain Adaptation for Machine Reading Comprehension](https://mlsanity.com/paper/1908.09209/) (2019)
- [Harvesting and Refining Question-Answer Pairs for Unsupervised QA](https://mlsanity.com/paper/2005.02925/) (2020)
- [Contrastive Domain Adaptation for Question Answering using Limited Text Corpora](https://mlsanity.com/paper/2108.13854/) (2021)

## Built on by (most cited first, 12 of 12)

- [ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples](https://mlsanity.com/paper/2210.12374/) (2022)
- [Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Survey](https://mlsanity.com/paper/2208.03197/) (2022)
- [Domain Adaptation for Question Answering via Question Classification](https://mlsanity.com/paper/2209.04998/) (2022)
- [Modeling What-to-ask and How-to-ask for Answer-unaware Conversational Question Generation](https://mlsanity.com/paper/2305.03088/) (2023)
- [Learning to Generalize for Cross-domain QA](https://mlsanity.com/paper/2305.08208/) (2023)
- [PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development](https://mlsanity.com/paper/2301.09715/) (2023)
- [Source-Free Domain Adaptation for Question Answering with Masked Self-training](https://mlsanity.com/paper/2212.09563/) (2022)
- [QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation](https://mlsanity.com/paper/2210.10861/) (2022)
- [Auto-GDA: Automatic Domain Adaptation for Efficient Grounding Verification in Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.03461/) (2024)
- [Birdie: Advancing State Space Models with Reward-Driven Objectives and Curricula](https://mlsanity.com/paper/2411.01030/) (2024)
- [DomainInv: Domain Invariant Fine Tuning and Adversarial Label Correction For QA Domain Adaptation](https://mlsanity.com/paper/2305.05589/) (2023)
- [Domain Adaptation of Multilingual Semantic Search -- Literature Review](https://mlsanity.com/paper/2402.02932/) (2024)

## Abstract (arXiv)

Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models. Since synthetic questions are often noisy in practice, existing work adapts scores from a pretrained QA (or QG) model as criteria to select high-quality questions. However, these scores do not directly serve the ultimate goal of improving QA performance on the target domain. In this paper, we introduce a novel idea of training a question value estimator (QVE) that directly estimates the usefulness of synthetic questions for improving the target-domain QA performance. By conducting comprehensive experiments, we show that the synthetic questions selected by QVE can help achieve better target-domain QA performance, in comparison with existing techniques. We additionally show that by using such questions and only around 15% of the human annotations on the target domain, we can achieve comparable performance to the fully-supervised baselines.