mlsanityData to 7 Oct 2026

Gradual Fine-Tuning for Low-Resource Domain Adaptation

arXiv 2103.02205 ↗First posted 3 Mar 2021cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Fine-tuning is known to improve NLP models by adapting an initial model trained on more plentiful but less domain-salient examples to data in a target domain. Such domain adaptation is typically done using one stage of fine-tuning. We demonstrate that gradually fine-tuning in a multi-stage process can yield substantial further gains and can be applied without modifying the model or learning objective.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

17 of 29 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  4. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  5. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  6. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  7. 2019Entity, Relation, and Event Extraction with Contextualized Span Representationscited by 261
  8. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
  9. 2019Uncertainty-based Continual Learning with Adaptive Regularizationcited by 144
  10. 2019SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Trackingcited by 63
  11. 2019Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 59
  12. 2020Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpuscited by 49
5 more
  1. 2019Improving Neural Machine Translation Robustness via Data Augmentation: Beyond Back Translationcited by 16
  2. 2019Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translationcited by 11
  3. 2020Cross-lingual Semantic Role Labeling with Model Transfercited by 11
  4. 2017Question Answering through Transfer Learning from Large Fine-grained Supervision Datacited by 9
  5. 2019Translate and Label! An Encoder-Decoder Approach for Cross-lingual Semantic Role Labelingcited by 7

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 8; 2023: 7; 2024: 4.

23 papers.

Sort:

  1. 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
  2. 2022Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extractioncited by 40
  3. 2021BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translationcited by 35
  4. 2022On the Domain Adaptation and Generalization of Pretrained Language Models: A Surveycited by 22
  5. 2021Moving on from OntoNotes: Coreference Resolution Model Transfercited by 18
  6. 2021Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extractioncited by 16
  7. 2021Language Model Priming for Cross-Lingual Event Extractioncited by 13
  8. 2023RadGraph2: Modeling Disease Progression in Radiology Reports via Hierarchical Information Extractioncited by 11
  9. 2022Por Qu'e N~ao Utiliser Alla Spraak? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfercited by 6
  10. 2023Domain Generalization with Fourier Transform and Soft Thresholdingcited by 4
  11. 2022Language Agnostic Code-Mixing Data Augmentation by Predicting Linguistic Patternscited by 3
  12. 2024Exploring Data-Efficient Adaptation of Large Language Models for Code Generationcited by 3
11 more
  1. 2024Leveraging Large Language Models for Code-Mixed Data Augmentation in Sentiment Analysiscited by 3
  2. 2023Simple yet Effective Code-Switching Language Identification with Multitask Pre-Training and Transfer Learningcited by 1
  3. 2023Multilingual Sentence-Level Semantic Search using Meta-Distillation Learningcited by 1
  4. 2024Improving Intersession Reproducibility for Forearm Ultrasound based Hand Gesture Classification through an Incremental Learning Approachcited by 1
  5. 2022Domain Adaptation in 3D Object Detection with Gradual Batch Alternation Trainingcited by 0
  6. 2022On Robust Incremental Learning over Many Multilingual Stepscited by 0
  7. 2022An Augmentation Strategy for Visually Rich Documentscited by 0
  8. 2023Impact of Subword Pooling Strategy on Cross-lingual Event Detectioncited by 0
  9. 2023Towards Flow Graph Prediction of Open-Domain Procedural Textscited by 0
  10. 2023Using Weak Supervision and Data Augmentation in Question Answeringcited by 0
  11. 2024Gradual Fine-Tuning with Graph Routing for Multi-Source Unsupervised Domain Adaptationcited by 0