# GradMix: Multi-source Transfer across Domains and Tasks

arXiv: https://arxiv.org/abs/2002.03264 · HTML: https://mlsanity.com/paper/2002.03264/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2020.

## Built on by (most cited first, 4 of 4)

- [Relation-aware Compositional Zero-shot Learning for Attribute-Object Pair Recognition](https://mlsanity.com/paper/2108.04603/) (2021)
- [Model ensemble instead of prompt fusion: a sample-specific knowledge transfer method for few-shot prompt tuning](https://mlsanity.com/paper/2210.12587/) (2022)
- [Learning to Minimize the Remainder in Supervised Learning](https://mlsanity.com/paper/2201.09193/) (2022)
- [Model Recycling Framework for Multi-Source Data-Free Supervised Transfer Learning](https://mlsanity.com/paper/2508.02039/) (2025)

## Abstract (arXiv)

The computer vision community is witnessing an unprecedented rate of new tasks being proposed and addressed, thanks to the deep convolutional networks' capability to find complex mappings from X to Y. The advent of each task often accompanies the release of a large-scale annotated dataset, for supervised training of deep network. However, it is expensive and time-consuming to manually label sufficient amount of training data. Therefore, it is important to develop algorithms that can leverage off-the-shelf labeled dataset to learn useful knowledge for the target task. While previous works mostly focus on transfer learning from a single source, we study multi-source transfer across domains and tasks (MS-DTT), in a semi-supervised setting. We propose GradMix, a model-agnostic method applicable to any model trained with gradient-based learning rule, to transfer knowledge via gradient descent by weighting and mixing the gradients from all sources during training. GradMix follows a meta-learning objective, which assigns layer-wise weights to the source gradients, such that the combined gradient follows the direction that minimize the loss for a small set of samples from the target dataset. In addition, we propose to adaptively adjust the learning rate for each mini-batch based on its importance to the target task, and a pseudo-labeling method to leverage the unlabeled samples in the target domain. We conduct MS-DTT experiments on two tasks: digit recognition and action recognition, and demonstrate the advantageous performance of the proposed method against multiple baselines.