mlsanityData to 7 Oct 2026

Learning to select data for transfer learning with Bayesian Optimization

arXiv 1707.05246 ↗First posted 17 Jul 2017cs.CL cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

Domain similarity measures can be used to gauge adaptability and select suitable data for transfer learning, but existing approaches define ad hoc measures that are deemed suitable for respective tasks. Inspired by work on curriculum learning, we propose to learn data selection measures using Bayesian Optimization and evaluate them across models, domains and tasks. Our learned measures outperform existing domain similarity measures significantly on three tasks: sentiment analysis, part-of-speech tagging, and parsing. We show the importance of complementing similarity with diversity, and that learned measures are -- to some degree -- transferable across models, domains, and even tasks.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 17; 2022: 10; 2023: 10; 2024: 17; 2025: 6; 2026: 3.

64 papers.

Sort:

By area:

  1. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  2. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  3. 2023Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Modelscited by 120
  4. 2021ClimateBert: A Pretrained Language Model for Climate-Related Textcited by 89
  5. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 86
  6. 2018Multi-Source Domain Adaptation with Mixture of Expertscited by 62
  7. 2025Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-trainingcited by 62
  8. 2023ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptationcited by 58
  9. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  10. 2021Adversarial Learning for Zero-Shot Stance Detection on Social Mediacited by 36
  11. 2023Efficient Continual Pre-training for Building Domain Specific Large Language Modelscited by 33
  12. 2022AdaPrompt: Adaptive Model Training for Prompt-based NLPcited by 30
38 more
  1. 2023Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptationcited by 30
  2. 2021Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Trainingcited by 28
  3. 2024TSDS: Data Selection for Task-Specific Model Finetuningcited by 28
  4. 2024Checkpoint Merging via Bayesian Optimization in LLM Pretrainingcited by 27
  5. 2024Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Textscited by 26
  6. 2021Focus on the Common Good: Group Distributional Robustness Followscited by 25
  7. 2024Task-Adaptive Pretrained Language Models via Clustered-Importance Samplingcited by 18
  8. 2021The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architecturescited by 16
  9. 2021Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimizationcited by 15
  10. 2022Synthetic Question Value Estimation for Domain Adaptation of Question Answeringcited by 12
  11. 2025Dynamic Fisher-weighted Model Merging via Bayesian Optimizationcited by 12
  12. 2021To Share or not to Share: Predicting Sets of Sources for Model Transfer Learningcited by 11
  13. 2021Evaluating the Values of Sources in Transfer Learningcited by 11
  14. 2022Active Learning Over Multiple Domains in Natural Language Taskscited by 11
  15. 2022Automatic Document Selection for Efficient Encoder Pretrainingcited by 11
  16. 2023NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysiscited by 10
  17. 2024Decomposing the Neurons: Activation Sparsity via Mixture of Experts for Continual Test Time Adaptationcited by 10
  18. 2021Learning Visual Models using a Knowledge Graph as a Trainercited by 8
  19. 2021Genre as Weak Supervision for Cross-lingual Dependency Parsingcited by 8
  20. 2021Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviewscited by 7
  21. 2021GradTS: A Gradient-Based Automatic Auxiliary Task Selection Method Based on Transformer Networkscited by 7
  22. 2022AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Modelscited by 7
  23. 2022A Survey on Visual Transfer Learning using Knowledge Graphscited by 7
  24. 2024How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?cited by 5
  25. 2024Improve Knowledge Distillation via Label Revision and Data Selectioncited by 5
  26. 2021How Universal is Genre in Universal Dependencies?cited by 4
  27. 2022Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answeringcited by 4
  28. 2022Navigating Memory Construction by Global Pseudo-Task Simulation for Continual Learningcited by 4
  29. 2022Rationale-Guided Few-Shot Classification to Detect Abusive Languagecited by 4
  30. 2024MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressionscited by 4
  31. 2022Analyzing Multi-Task Learning for Abstractive Text Summarizationcited by 3
  32. 2024Selecting Subsets of Source Data for Transfer Learning with Applications in Metal Additive Manufacturingcited by 3
  33. 2024Rethinking Performance Analysis for Configurable Software Systems: A Case Study from a Fitness Landscape Perspectivecited by 3
  34. 2023A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasetscited by 2
  35. 2023Towards Robust and Efficient Continual Language Learningcited by 2
  36. 2024Can Humans Identify Domains?cited by 2
  37. 2024Proxy-informed Bayesian transfer learning with unknown sourcescited by 2
  38. 2025QUEST: Quality-aware Semi-supervised Table Extraction for Business Documentscited by 2

50 most cited of 64 All 64 (JSON)