# From Multi-label Learning to Cross-Domain Transfer: A Model-Agnostic Approach

arXiv: https://arxiv.org/abs/2207.11742 · HTML: https://mlsanity.com/paper/2207.11742/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 69 references that are in the map.
- First posted on arXiv 24 Jul 2022.

## Builds on (most cited first, 23 of 23)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [Training independent subnetworks for robust prediction](https://mlsanity.com/paper/2010.06610/) (2020)
- [Learning to cluster in order to transfer across domains and tasks](https://mlsanity.com/paper/1711.10125/) (2017)
- [Modular meta-learning](https://mlsanity.com/paper/1806.10166/) (2018)
- [Comprehensive Comparative Study of Multi-Label Classification Methods](https://mlsanity.com/paper/2102.07113/) (2021)
- [Modular Meta-Learning with Shrinkage](https://mlsanity.com/paper/1909.05557/) (2019)
- [Classifier Chains: A Review and Perspectives](https://mlsanity.com/paper/1912.13405/) (2019)
- [Learning Deep Latent Spaces for Multi-Label Classification](https://mlsanity.com/paper/1707.00418/) (2017)
- [Optimal Bayesian Transfer Learning](https://mlsanity.com/paper/1801.00857/) (2018)
- [Multi-Target Prediction: A Unifying View on Problems and Methods](https://mlsanity.com/paper/1809.02352/) (2018)
- [Handling Concept Drift via Model Reuse](https://mlsanity.com/paper/1809.02804/) (2018)
- [Multi-Target Regression via Random Linear Target Combinations](https://mlsanity.com/paper/1404.5065/) (2014)
- [Rectifying Classifier Chains for Multi-Label Classification](https://mlsanity.com/paper/1906.02915/) (2019)
- [Deep tree-ensembles for multi-output prediction](https://mlsanity.com/paper/2011.02829/) (2020)
- [Multi-label Classification using Labels as Hidden Nodes](https://mlsanity.com/paper/1503.09022/) (2015)

## Built on by (most cited first, 3 of 3)

- [In-context Learning of Evolving Data Streams with Tabular Foundational Models](https://mlsanity.com/paper/2502.16840/) (2025)
- [Adapting Differential Molecular Representation with Hierarchical Prompts for Multi-label Property Prediction](https://mlsanity.com/paper/2405.18724/) (2024)
- [PET/CT Radiogenomic Mutation Prediction in Non-Small Cell Lung Cancer Using Multi-Label Learning](https://mlsanity.com/paper/2608.09721/) (2026)

## Abstract (arXiv)

In multi-label learning, a particular case of multi-task learning where a single data point is associated with multiple target labels, it was widely assumed in the literature that, to obtain best accuracy, the dependence among the labels should be explicitly modeled. This premise led to a proliferation of methods offering techniques to learn and predict labels together, for example where the prediction for one label influences predictions for other labels. Even though it is now acknowledged that in many contexts a model of dependence is not required for optimal performance, such models continue to outperform independent models in some of those very contexts, suggesting alternative explanations for their performance beyond label dependence, which the literature is only recently beginning to unravel. Leveraging and extending recent discoveries, we turn the original premise of multi-label learning on its head, and approach the problem of joint-modeling specifically under the absence of any measurable dependence among task labels; for example, when task labels come from separate problem domains. We shift insights from this study towards building an approach for transfer learning that challenges the long-held assumption that transferability of tasks comes from measurements of similarity between the source and target domains or models. This allows us to design and test a method for transfer learning, which is model driven rather than purely data driven, and furthermore it is black box and model-agnostic (any base model class can be considered). We show that essentially we can create task-dependence based on source-model capacity. The results we obtain have important implications and provide clear directions for future work, both in the areas of multi-label and transfer learning.