# Gradual Fine-Tuning with Graph Routing for Multi-Source Unsupervised Domain Adaptation

arXiv: https://arxiv.org/abs/2411.07185 · HTML: https://mlsanity.com/paper/2411.07185/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [Adversarial Multi-task Learning for Text Classification](https://mlsanity.com/paper/1704.05742/) (2017)
- [Maximum Density Divergence for Domain Adaptation](https://mlsanity.com/paper/2004.12615/) (2020)
- [Faster Wasserstein Distance Estimation with the Sinkhorn Divergence](https://mlsanity.com/paper/2006.08172/) (2020)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [Boosting Domain Adaptation by Discovering Latent Domains](https://mlsanity.com/paper/1805.01386/) (2018)
- [Understanding Gradual Domain Adaptation: Improved Analysis, Optimal Path and Beyond](https://mlsanity.com/paper/2204.08200/) (2022)
- [Gradual Domain Adaptation without Indexed Intermediate Domains](https://mlsanity.com/paper/2207.04587/) (2022)
- [Domain Divergences: a Survey and Empirical Analysis](https://mlsanity.com/paper/2010.12198/) (2020)
- [Gradual Fine-Tuning for Low-Resource Domain Adaptation](https://mlsanity.com/paper/2103.02205/) (2021)
- [Evaluating the Values of Sources in Transfer Learning](https://mlsanity.com/paper/2104.12567/) (2021)
- [Asymptotic sequential Rademacher complexity of a finite function class](https://mlsanity.com/paper/1605.03843/) (2016)

## Abstract (arXiv)

Multi-source unsupervised domain adaptation aims to leverage labeled data from multiple source domains for training a machine learning model to generalize well on a target domain without labels. Source domain selection plays a crucial role in determining the model's performance. It relies on the similarities amongst source and target domains. Nonetheless, existing work for source domain selection often involves heavyweight computational procedures, especially when dealing with numerous source domains and the need to identify the best ones from them. In this paper, we introduce a framework for gradual fine tuning (GFT) of machine learning models on multiple source domains. We represent multiple source domains as an undirected weighted graph. We then give a new generalization error bound for GFT along any path within the graph, which is used to determine the optimal path corresponding to the optimal training order. With this formulation, we introduce three lightweight graph-routing strategies which tend to minimize the error bound. Our best strategy improves 2.3% of accuracy over the state-of-the-art on Natural Language Inference (NLI) task and achieves competitive performance on Sentiment Analysis (SA) task, especially a 3.9% improvement on a more diverse subset of data we use for SA.