# Identifying Suitable Tasks for Inductive Transfer Through the Analysis of Feature Attributions

arXiv: https://arxiv.org/abs/2202.01096 · HTML: https://mlsanity.com/paper/2202.01096/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Benchmarking Deep Learning Interpretability in Time Series Predictions](https://mlsanity.com/paper/2010.13924/) (2020)
- [How Important Is a Neuron?](https://mlsanity.com/paper/1805.12233/) (2018)
- [Identifying beneficial task relations for multi-task learning in deep neural networks](https://mlsanity.com/paper/1702.08303/) (2017)
- [Efficient Saliency Maps for Explainable AI](https://mlsanity.com/paper/1911.11293/) (2019)
- [Explain and Predict, and then Predict Again](https://mlsanity.com/paper/2101.04109/) (2021)
- [How Transferable are Neural Networks in NLP Applications?](https://mlsanity.com/paper/1603.06111/) (2016)
- [Explainable Recommender Systems via Resolving Learning Representations](https://mlsanity.com/paper/2008.09316/) (2020)

## Abstract (arXiv)

Transfer learning approaches have shown to significantly improve performance on downstream tasks. However, it is common for prior works to only report where transfer learning was beneficial, ignoring the significant trial-and-error required to find effective settings for transfer. Indeed, not all task combinations lead to performance benefits, and brute-force searching rapidly becomes computationally infeasible. Hence the question arises, can we predict whether transfer between two tasks will be beneficial without actually performing the experiment? In this paper, we leverage explainability techniques to effectively predict whether task pairs will be complementary, through comparison of neural network activation between single-task models. In this way, we can avoid grid-searches over all task and hyperparameter combinations, dramatically reducing the time needed to find effective task pairs. Our results show that, through this approach, it is possible to reduce training time by up to 83.5% at a cost of only 0.034 reduction in positive-class F1 on the TREC-IS 2020-A dataset.