mlsanityData to 13 Oct 2026

Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answering

arXiv 2306.06779 ↗First posted 11 Jun 2023cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

In this work, we study multi-source test-time model adaptation from user feedback, where K distinct models are established for adaptation. To allow efficient adaptation, we cast the problem as a stochastic decision-making process, aiming to determine the best adapted model after adaptation. We discuss two frameworks: multi-armed bandit learning and multi-armed dueling bandits. Compared to multi-armed bandit learning, the dueling framework allows pairwise collaboration among K models, which is solved by a novel method named Co-UCB proposed in this work. Experiments on six datasets of extractive question answering (QA) show that the dueling framework using Co-UCB is more effective than other strong baselines for our studied problem.

Key facts. Cited by 6 papers in the map (data through 13 Oct 2026). Builds on 39 of its 58 references that are in the map. First posted on arXiv 11 Jun 2023.

Builds on

39 of 58 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2023GPT-4 Technical Reportcited by 18,278
  4. 2022Training language models to follow instructions with human feedbackcited by 15,856
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,213
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,133
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,916
  8. 2017Deep reinforcement learning from human preferencescited by 3,871
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,925
  11. 2020Learning to summarize from human feedbackcited by 2,575
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,229
27 more
  1. 2020Meta-Learning in Neural Networks: A Surveycited by 945
  2. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 833
  3. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 785
  4. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 741
  5. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  6. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 222
  7. 2020Universal Source-Free Domain Adaptationcited by 219
  8. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  9. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  10. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 144
  11. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  12. 2021Unsupervised Multi-source Domain Adaptation Without Access to Source Datacited by 76
  13. 2017Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedbackcited by 71
  14. 2014Algorithms for multi-armed bandit problemscited by 64
  15. 2018Multi-Source Domain Adaptation with Mixture of Expertscited by 62
  16. 2017Multi-dueling Bandits with Dependent Armscited by 35
  17. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  18. 2018Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedbackcited by 32
  19. 2020Multi-Source Domain Adaptation for Text Classification via DistanceNet-Banditscited by 30
  20. 2021Efficient Test Time Adapter Ensembling for Low-resource Language Varietiescited by 26
  21. 2020An Imitation Game for Learning Semantic Parsers from User Interactioncited by 23
  22. 2022Simulating Bandit Learning from User Feedback for Extractive Question Answeringcited by 14
  23. 2018APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learningcited by 9
  24. 2020Learning a Cost-Effective Annotation Policy for Question Answeringcited by 9
  25. 2023Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Studycited by 8
  26. 2021Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effortcited by 7
  27. 2019Self-Regulated Interactive Sequence-to-Sequence Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 1.

6 papers.

  1. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  2. 2023Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Modelscited by 37
  3. 2024Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Modelscited by 18
  4. 2024PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answeringcited by 15
  5. 2024Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question Answeringcited by 9
  6. 2024Plug-and-Play Transformer Modules for Test-Time Adaptationcited by 0