# Supervised and Unsupervised Transfer Learning for Question Answering

arXiv: https://arxiv.org/abs/1711.05345 · HTML: https://mlsanity.com/paper/1711.05345/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2017.

## Built on by (most cited first, 24 of 24)

- [L-Eval: Instituting Standardized Evaluation for Long Context Language Models](https://mlsanity.com/paper/2307.11088/) (2023)
- [Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum](https://mlsanity.com/paper/2405.13226/) (2024)
- [Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask](https://mlsanity.com/paper/2204.02908/) (2022)
- [Efficiently Fusing Pretrained Acoustic and Linguistic Encoders for Low-resource Speech Recognition](https://mlsanity.com/paper/2101.06699/) (2021)
- [Mixture of In-Context Experts Enhance LLMs' Long Context Awareness](https://mlsanity.com/paper/2406.19598/) (2024)
- [LibriSQA: A Novel Dataset and Framework for Spoken Question Answering with Large Language Models](https://mlsanity.com/paper/2308.10390/) (2023)
- [Synthetic Question Value Estimation for Domain Adaptation of Question Answering](https://mlsanity.com/paper/2203.08926/) (2022)
- [Back-Training excels Self-Training at Unsupervised Domain Adaptation of Question Generation and Passage Retrieval](https://mlsanity.com/paper/2104.08801/) (2021)
- [Contextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge](https://mlsanity.com/paper/2104.05216/) (2021)
- [Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data](https://mlsanity.com/paper/2102.01226/) (2021)
- [Improving Question Answering Performance Using Knowledge Distillation and Active Learning](https://mlsanity.com/paper/2109.12662/) (2021)
- [Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention](https://mlsanity.com/paper/2312.08618/) (2023)
- [SkyLadder: Better and Faster Pretraining via Context Window Scheduling](https://mlsanity.com/paper/2503.15450/) (2025)
- [Analyzing Multiple-Choice Reading and Listening Comprehension Tests](https://mlsanity.com/paper/2307.01076/) (2023)
- [M-SENSE: Modeling Narrative Structure in Short Personal Narratives Using Protagonist's Mental Representations](https://mlsanity.com/paper/2302.09418/) (2023)
- [RSQ: Learning from Important Tokens Leads to Better Quantized LLMs](https://mlsanity.com/paper/2503.01820/) (2025)
- [Auto-Transfer: Learning to Route Transferrable Representations](https://mlsanity.com/paper/2202.01011/) (2022)
- [QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation](https://mlsanity.com/paper/2210.10861/) (2022)
- [Multi-Facet Blending for Faceted Query-by-Example Retrieval](https://mlsanity.com/paper/2412.01443/) (2024)
- [Zero-Shot Estimation of Base Models' Weights in Ensemble of Machine Reading Comprehension Systems for Robust Generalization](https://mlsanity.com/paper/2106.16013/) (2021)
- [An Initial Investigation of Non-Native Spoken Question-Answering](https://mlsanity.com/paper/2107.04691/) (2021)
- [Where Was COVID-19 First Discovered? Designing a Question-Answering System for Pandemic Situations](https://mlsanity.com/paper/2204.08787/) (2022)
- [Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systems](https://mlsanity.com/paper/2107.00315/) (2021)
- [Is Robustness Transferable across Languages in Multilingual Neural Machine Translation?](https://mlsanity.com/paper/2310.20162/) (2023)

## Abstract (arXiv)

Although transfer learning has been shown to be successful for tasks like object and speech recognition, its applicability to question answering (QA) has yet to be well-studied. In this paper, we conduct extensive experiments to investigate the transferability of knowledge learned from a source QA dataset to a target dataset using two QA models. The performance of both models on a TOEFL listening comprehension test (Tseng et al., 2016) and MCTest (Richardson et al., 2013) is significantly improved via a simple transfer learning technique from MovieQA (Tapaswi et al., 2016). In particular, one of the models achieves the state-of-the-art on all target datasets; for the TOEFL listening comprehension test, it outperforms the previous best model by 7%. Finally, we show that transfer learning is helpful even in unsupervised scenarios when correct answers for target QA dataset examples are not available.