# Deep Model Transferability from Attribution Maps

arXiv: https://arxiv.org/abs/1909.11902 · HTML: https://mlsanity.com/paper/1909.11902/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2019.

## Built on by (most cited first, 28 of 28)

- [Deep Model Reassembly](https://mlsanity.com/paper/2210.17409/) (2022)
- [Scalable Diverse Model Selection for Accessible Transfer Learning](https://mlsanity.com/paper/2111.06977/) (2021)
- [Model Spider: Learning to Rank Pre-Trained Models Efficiently](https://mlsanity.com/paper/2306.03900/) (2023)
- [Meta Discovery: Learning to Discover Novel Classes given Very Limited Data](https://mlsanity.com/paper/2102.04002/) (2021)
- [How stable are Transferability Metrics evaluations?](https://mlsanity.com/paper/2204.01403/) (2022)
- [Transferability Metrics for Selecting Source Model Ensembles](https://mlsanity.com/paper/2111.13011/) (2021)
- [Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languages](https://mlsanity.com/paper/2205.12676/) (2022)
- [TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation](https://mlsanity.com/paper/2106.06326/) (2021)
- [UniNet: A Unified Scene Understanding Network and Exploring Multi-Task Relationships through the Lens of Adversarial Attacks](https://mlsanity.com/paper/2108.04584/) (2021)
- [Model Doctor: A Simple Gradient Aggregation Strategy for Diagnosing and Treating CNN Classifiers](https://mlsanity.com/paper/2112.04934/) (2021)
- [Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approach](https://mlsanity.com/paper/2309.02429/) (2023)
- [ModelGiF: Gradient Fields for Model Functional Distance](https://mlsanity.com/paper/2309.11013/) (2023)
- [Model Selection with Model Zoo via Graph Learning](https://mlsanity.com/paper/2404.03988/) (2024)
- [The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning](https://mlsanity.com/paper/2406.11721/) (2024)
- [Understanding Knowledge Transferability for Transfer Learning: A Survey](https://mlsanity.com/paper/2507.03175/) (2025)
- [Exploiting a Zoo of Checkpoints for Unseen Tasks](https://mlsanity.com/paper/2111.03628/) (2021)
- [Transferability Estimation Based On Principal Gradient Expectation](https://mlsanity.com/paper/2211.16299/) (2022)
- [Visual Boundary Knowledge Translation for Foreground Segmentation](https://mlsanity.com/paper/2108.00379/) (2021)
- [Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks](https://mlsanity.com/paper/2203.13453/) (2022)
- [Federated Selective Aggregation for Knowledge Amalgamation](https://mlsanity.com/paper/2207.13309/) (2022)
- ["It's a Match!" -- A Benchmark of Task Affinity Scores for Joint Learning](https://mlsanity.com/paper/2301.02873/) (2023)
- [Pre-Trained Model Recommendation for Downstream Fine-tuning](https://mlsanity.com/paper/2403.06382/) (2024)
- [k-NN as a Simple and Effective Estimator of Transferability](https://mlsanity.com/paper/2503.18528/) (2025)
- [Match & Choose: Model Selection Framework for Fine-tuning Text-to-Image Diffusion Models](https://mlsanity.com/paper/2508.10993/) (2025)
- [Towards Estimating Transferability using Hard Subsets](https://mlsanity.com/paper/2301.06928/) (2023)
- [Graph Neural Networks-based Hybrid Framework For Predicting Particle Crushing Strength](https://mlsanity.com/paper/2307.13909/) (2023)
- [Efficient Transferability Assessment for Selection of Pre-trained Detectors](https://mlsanity.com/paper/2403.09432/) (2024)
- [Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression](https://mlsanity.com/paper/2606.18304/) (2026)

## Abstract (arXiv)

Exploring the transferability between heterogeneous tasks sheds light on their intrinsic interconnections, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter. In this paper, we propose an embarrassingly simple yet very efficacious approach to estimating the transferability of deep networks, especially those handling vision tasks. Unlike the seminal work of taskonomy that relies on a large number of annotations as supervision and is thus computationally cumbersome, the proposed approach requires no human annotations and imposes no constraints on the architectures of the networks. This is achieved, specifically, via projecting deep networks into a model space, wherein each network is treated as a point and the distances between two points are measured by deviations of their produced attribution maps. The proposed approach is several-magnitude times faster than taskonomy, and meanwhile preserves a task-wise topological structure highly similar to the one obtained by taskonomy. Code is available at https://github.com/zju-vipa/TransferbilityFromAttributionMaps.