mlsanityData to 7 Oct 2026

Active Learning Over Multiple Domains in Natural Language Tasks

arXiv 2202.00254 ↗First posted 1 Feb 2022cs.CL cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Studies of active learning traditionally assume the target and source data stem from a single domain. However, in realistic applications, practitioners often require active learning with multiple sources of out-of-distribution data, where it is unclear a priori which data sources will help or hurt the target domain. We survey a wide variety of techniques in active learning (AL), domain shift detection (DS), and multi-domain sampling to examine this challenging setting for question answering and sentiment analysis. We ask (1) what family of methods are effective for this task? And, (2) what properties of selected examples and domains achieve strong results? Among 18 acquisition functions from 4 families of methods, we find H-Divergence methods, and particularly our proposed variant DAL-E, yield effective results, averaging 2-3% improvements over the random baseline. We also show the importance of a diverse allocation of domains, as well as room-for-improvement of existing methods on both domain and example selection. Our findings yield the first comprehensive analysis of both existing and novel methods for practitioners faced with multi-domain active learning for natural language tasks.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 1 Feb 2022.

Builds on

33 of 54 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  11. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,837
21 more
  1. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  2. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  3. 2017Deep Bayesian Active Learning with Image Datacited by 812
  4. 2016NewsQA: A Machine Comprehension Datasetcited by 330
  5. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 240
  6. 2020Selective Question Answering under Domain Shiftcited by 228
  7. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  8. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  9. 2019A review of domain adaptation without target labelscited by 154
  10. 2019Discriminative Active Learningcited by 97
  11. 2018Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Studycited by 90
  12. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
  13. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  14. 2018Practical Obstacles to Deploying Active Learningcited by 68
  15. 2017Learning to select data for transfer learning with Bayesian Optimizationcited by 64
  16. 2019MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehensioncited by 60
  17. 2018Strong Baselines for Neural Semi-supervised Learning under Domain Shiftcited by 39
  18. 2020Active Learning under Label Shiftcited by 23
  19. 2019An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answeringcited by 22
  20. 2020MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scalecited by 20
  21. 2020Recommendation Chart of Domains for Cross-Domain Sentiment Analysis:Findings of A 20 Domain Studycited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 6; 2024: 1; 2025: 2.

11 papers.

  1. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  2. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  3. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  4. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  5. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  6. 2023On the Limitations of Simulating Active Learningcited by 9
  7. 2023Investigating Multi-source Active Learning for Natural Language Inferencecited by 6
  8. 2023DeMuX: Data-efficient Multilingual Learningcited by 1
  9. 2023Semi-Automated Construction of Food Composition Knowledge Basecited by 0
  10. 2025Active Learning Methods for Efficient Data Utilization and Model Performance Enhancementcited by 0
  11. 2025Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibrationcited by 0