mlsanityData to 7 Oct 2026

Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Study

arXiv 2302.04618 ↗First posted 9 Feb 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

A deployed question answering (QA) model can easily fail when the test data has a distribution shift compared to the training data. Robustness tuning (RT) methods have been widely studied to enhance model robustness against distribution shifts before model deployment. However, can we improve a model after deployment? To answer this question, we evaluate test-time adaptation (TTA) to improve a model after deployment. We first introduce COLDQA, a unified evaluation benchmark for robust QA against text corruption and changes in language and domain. We then evaluate previous TTA methods on COLDQA and compare them to RT methods. We also propose a novel TTA method called online imitation learning (OIL). Through extensive experiments, we find that TTA is comparable to RT methods, and applying TTA after RT can significantly boost the performance on COLDQA. Our proposed OIL improves TTA to be more robust to variation in hyper-parameters and test distributions over time.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 9 Feb 2023.

Builds on

40 of 57 references.

References · 40

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  9. 2020Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptationcited by 844
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  11. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
  12. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
28 more
  1. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  2. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  3. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  4. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  5. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  6. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  7. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  8. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  9. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  10. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 142
  11. 2020An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Modelscited by 133
  12. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  13. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  14. 2021MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaptioncited by 65
  15. 2021PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domainscited by 61
  16. 2021Consistency Regularization for Cross-Lingual Fine-Tuningcited by 41
  17. 2021Multi-view Subword Regularizationcited by 40
  18. 2020RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systemscited by 36
  19. 2021UDALM: Unsupervised Domain Adaptation through Language Modelingcited by 33
  20. 2020Robustness Testing of Language Understanding in Task-Oriented Dialogcited by 32
  21. 2021Efficient Test Time Adapter Ensembling for Low-resource Language Varietiescited by 26
  22. 2021NoiseQA: Challenge Set Evaluation for User-Centric Question Answeringcited by 25
  23. 2022Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deploymentcited by 25
  24. 2020Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Modelcited by 24
  25. 2021Self-Supervised Test-Time Learning for Reading Comprehensioncited by 22
  26. 2022On Continual Model Refinement in Out-of-Distribution Data Streamscited by 22
  27. 2020Posterior Differential Regularization with f-divergence for Improving Model Robustnesscited by 20
  28. 2022Simulating Bandit Learning from User Feedback for Extractive Question Answeringcited by 14

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 0; 2025: 3.

8 papers.

  1. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  2. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  3. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  4. 2023Test-Time Self-Adaptive Small Language Models for Question Answeringcited by 7
  5. 2023Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answeringcited by 6
  6. 2025Harnessing Test-time Adaptation for NLU tasks Involving Dialects of Englishcited by 1
  7. 2025CTTA-T: Continual Test-Time Adaptation for Text Understanding via Teacher-Student with a Domain-aware and Generalized Teachercited by 1
  8. 2023Test-Time Adaptation with Perturbation Consistency Learningcited by 0