# DEPARA: Deep Attribution Graph for Deep Knowledge Transferability

arXiv: https://arxiv.org/abs/2003.07496 · HTML: https://mlsanity.com/paper/2003.07496/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2020.

## Built on by (most cited first, 18 of 18)

- [Frustratingly Easy Transferability Estimation](https://mlsanity.com/paper/2106.09362/) (2021)
- [Model Spider: Learning to Rank Pre-Trained Models Efficiently](https://mlsanity.com/paper/2306.03900/) (2023)
- [Meta Discovery: Learning to Discover Novel Classes given Very Limited Data](https://mlsanity.com/paper/2102.04002/) (2021)
- [How stable are Transferability Metrics evaluations?](https://mlsanity.com/paper/2204.01403/) (2022)
- [Transferability Metrics for Selecting Source Model Ensembles](https://mlsanity.com/paper/2111.13011/) (2021)
- [TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation](https://mlsanity.com/paper/2106.06326/) (2021)
- [Understanding the Transferability of Representations via Task-Relatedness](https://mlsanity.com/paper/2307.00823/) (2023)
- [Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representation](https://mlsanity.com/paper/2201.11576/) (2022)
- [UniNet: A Unified Scene Understanding Network and Exploring Multi-Task Relationships through the Lens of Adversarial Attacks](https://mlsanity.com/paper/2108.04584/) (2021)
- [Graph-Based Similarity of Neural Network Representations](https://mlsanity.com/paper/2111.11165/) (2021)
- [Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approach](https://mlsanity.com/paper/2309.02429/) (2023)
- [ModelGiF: Gradient Fields for Model Functional Distance](https://mlsanity.com/paper/2309.11013/) (2023)
- [Transferability Estimation Based On Principal Gradient Expectation](https://mlsanity.com/paper/2211.16299/) (2022)
- [Transferability Metrics for Object Detection](https://mlsanity.com/paper/2306.15306/) (2023)
- [Pre-Trained Model Recommendation for Downstream Fine-tuning](https://mlsanity.com/paper/2403.06382/) (2024)
- [k-NN as a Simple and Effective Estimator of Transferability](https://mlsanity.com/paper/2503.18528/) (2025)
- [Towards Estimating Transferability using Hard Subsets](https://mlsanity.com/paper/2301.06928/) (2023)
- [Efficient Transferability Assessment for Selection of Pre-trained Detectors](https://mlsanity.com/paper/2403.09432/) (2024)

## Abstract (arXiv)

Exploring the intrinsic interconnections between the knowledge encoded in PRe-trained Deep Neural Networks (PR-DNNs) of heterogeneous tasks sheds light on their mutual transferability, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter. In this paper, we propose the DEeP Attribution gRAph (DEPARA) to investigate the transferability of knowledge learned from PR-DNNs. In DEPARA, nodes correspond to the inputs and are represented by their vectorized attribution maps with regards to the outputs of the PR-DNN. Edges denote the relatedness between inputs and are measured by the similarity of their features extracted from the PR-DNN. The knowledge transferability of two PR-DNNs is measured by the similarity of their corresponding DEPARAs. We apply DEPARA to two important yet under-studied problems in transfer learning: pre-trained model selection and layer selection. Extensive experiments are conducted to demonstrate the effectiveness and superiority of the proposed method in solving both these problems. Code, data and models reproducing the results in this paper are available at urlhttps://github.com/zju-vipa/DEPARA.