# Effective Transfer Learning for Identifying Similar Questions: Matching User Questions to COVID-19 FAQs

arXiv: https://arxiv.org/abs/2008.13546 · HTML: https://mlsanity.com/paper/2008.13546/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2020.

## Built on by (most cited first, 30 of 48)

- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models](https://mlsanity.com/paper/2203.06904/) (2022)
- [CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP](https://mlsanity.com/paper/2104.08835/) (2021)
- [Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations](https://mlsanity.com/paper/2205.12685/) (2022)
- [Overthinking the Truth: Understanding how Language Models Process False Demonstrations](https://mlsanity.com/paper/2307.09476/) (2023)
- [Instruction Pre-Training: Language Models are Supervised Multitask Learners](https://mlsanity.com/paper/2406.14491/) (2024)
- [Active Learning Principles for In-Context Learning with Large Language Models](https://mlsanity.com/paper/2305.14264/) (2023)
- [Self-Guard: Empower the LLM to Safeguard Itself](https://mlsanity.com/paper/2310.15851/) (2023)
- [In-Context Learning Learns Label Relationships but Is Not Conventional Learning](https://mlsanity.com/paper/2307.12375/) (2023)
- [Adapting Large Language Models to Domains via Reading Comprehension](https://mlsanity.com/paper/2309.09530/) (2023)
- [BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers](https://mlsanity.com/paper/2404.18443/) (2024)
- [BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing](https://mlsanity.com/paper/2206.15076/) (2022)
- [Exploring Universal Intrinsic Task Subspace via Prompt Tuning](https://mlsanity.com/paper/2110.07867/) (2021)
- [MedS3: Towards Medical Slow Thinking with Self-Evolved Soft Dual-sided Process Supervision](https://mlsanity.com/paper/2501.12051/) (2025)
- [Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models](https://mlsanity.com/paper/2311.00287/) (2023)
- [Estimating the Hallucination Rate of Generative AI](https://mlsanity.com/paper/2406.07457/) (2024)
- [MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation](https://mlsanity.com/paper/2310.14088/) (2023)
- [Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack](https://mlsanity.com/paper/2407.16695/) (2024)
- [Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Experts](https://mlsanity.com/paper/2205.12701/) (2022)
- [Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?](https://mlsanity.com/paper/2302.08143/) (2023)
- [Improving Domain Adaptation through Extended-Text Reading Comprehension](https://mlsanity.com/paper/2401.07284/) (2024)
- [Exploring Demonstration Ensembling for In-context Learning](https://mlsanity.com/paper/2308.08780/) (2023)
- [Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning](https://mlsanity.com/paper/2404.07546/) (2024)
- [MedINST: Meta Dataset of Biomedical Instructions](https://mlsanity.com/paper/2410.13458/) (2024)
- [Few-shot Adaptation Works with UnpredicTable Data](https://mlsanity.com/paper/2208.01009/) (2022)
- [When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation](https://mlsanity.com/paper/2203.09391/) (2022)
- [Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning](https://mlsanity.com/paper/2312.17055/) (2023)
- [KisanQRS: A Deep Learning-based Automated Query-Response System for Agricultural Decision-Making](https://mlsanity.com/paper/2411.08883/) (2024)
- [Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels](https://mlsanity.com/paper/2409.12425/) (2024)

## Abstract (arXiv)

People increasingly search online for answers to their medical questions but the rate at which medical questions are asked online significantly exceeds the capacity of qualified people to answer them. This leaves many questions unanswered or inadequately answered. Many of these questions are not unique, and reliable identification of similar questions would enable more efficient and effective question answering schema. COVID-19 has only exacerbated this problem. Almost every government agency and healthcare organization has tried to meet the informational need of users by building online FAQs, but there is no way for people to ask their question and know if it is answered on one of these pages. While many research efforts have focused on the problem of general question similarity, these approaches do not generalize well to domains that require expert knowledge to determine semantic similarity, such as the medical domain. In this paper, we show how a double fine-tuning approach of pretraining a neural network on medical question-answer pairs followed by fine-tuning on medical question-question pairs is a particularly useful intermediate task for the ultimate goal of determining medical question similarity. While other pretraining tasks yield an accuracy below 78.7% on this task, our model achieves an accuracy of 82.6% with the same number of training examples, an accuracy of 80.0% with a much smaller training set, and an accuracy of 84.5% when the full corpus of medical question-answer data is used. We also describe a currently live system that uses the trained model to match user questions to COVID-related FAQs.