# Transferability Estimation using Bhattacharyya Class Separability

arXiv: https://arxiv.org/abs/2111.12780 · HTML: https://mlsanity.com/paper/2111.12780/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 75 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images](https://mlsanity.com/paper/1905.12886/) (2019)
- [What makes ImageNet good for transfer learning?](https://mlsanity.com/paper/1608.08614/) (2016)
- [LEEP: A New Measure to Evaluate Transferability of Learned Representations](https://mlsanity.com/paper/2002.12462/) (2020)
- [Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning](https://mlsanity.com/paper/1806.06193/) (2018)
- [LogME: Practical Assessment of Pre-trained Models for Transfer Learning](https://mlsanity.com/paper/2102.11005/) (2021)

## Built on by (most cited first, 30 of 54)

- [Selecting Large Language Model to Fine-tune via Rectified Scaling Law](https://mlsanity.com/paper/2402.02314/) (2024)
- [Model Spider: Learning to Rank Pre-Trained Models Efficiently](https://mlsanity.com/paper/2306.03900/) (2023)
- [How stable are Transferability Metrics evaluations?](https://mlsanity.com/paper/2204.01403/) (2022)
- [No Reason for No Supervision: Improved Generalization in Supervised Models](https://mlsanity.com/paper/2206.15369/) (2022)
- [How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?](https://mlsanity.com/paper/2501.11253/) (2025)
- [ETran: Energy-Based Transferability Estimation](https://mlsanity.com/paper/2308.02027/) (2023)
- [Foundation Model is Efficient Multimodal Multitask Model Selector](https://mlsanity.com/paper/2308.06262/) (2023)
- [Which Model to Transfer? A Survey on Transferability Estimation](https://mlsanity.com/paper/2402.15231/) (2024)
- [LOVM: Language-Only Vision Model Selection](https://mlsanity.com/paper/2306.08893/) (2023)
- [A Bag-of-Prototypes Representation for Dataset-Level Applications](https://mlsanity.com/paper/2303.13251/) (2023)
- [Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation](https://mlsanity.com/paper/2307.11958/) (2023)
- [Exploring Model Transferability through the Lens of Potential Energy](https://mlsanity.com/paper/2308.15074/) (2023)
- [End-to-End Supervised Multilabel Contrastive Learning](https://mlsanity.com/paper/2307.03967/) (2023)
- [Evidence > Intuition: Transferability Estimation for Encoder Selection](https://mlsanity.com/paper/2210.11255/) (2022)
- [The Performance of Transferability Metrics does not Translate to Medical Tasks](https://mlsanity.com/paper/2308.07444/) (2023)
- [Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approach](https://mlsanity.com/paper/2309.02429/) (2023)
- [Sequence Transferability and Task Order Selection in Continual Learning](https://mlsanity.com/paper/2502.06544/) (2025)
- [Exploring Structural Degradation in Dense Representations for Self-supervised Learning](https://mlsanity.com/paper/2510.17299/) (2025)
- [A Universal Metric of Dataset Similarity for Cross-silo Federated Learning](https://mlsanity.com/paper/2404.18773/) (2024)
- [Transferability Estimation Based On Principal Gradient Expectation](https://mlsanity.com/paper/2211.16299/) (2022)
- [Your representations are in the network: composable and parallel adaptation for large scale models](https://mlsanity.com/paper/2303.04105/) (2023)
- [A New Probabilistic Distance Metric With Application In Gaussian Mixture Reduction](https://mlsanity.com/paper/2306.07309/) (2023)
- [LEAD: Exploring Logit Space Evolution for Model Selection](https://mlsanity.com/paper/2507.14559/) (2025)
- [One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo](https://mlsanity.com/paper/2509.04208/) (2025)
- [On the pitfalls of entropy-based uncertainty for multi-class semi-supervised segmentation](https://mlsanity.com/paper/2203.03587/) (2022)
- [Feasibility of Transfer Learning: A Mathematical Framework](https://mlsanity.com/paper/2305.12985/) (2023)
- [Feature Space Perturbation: A Panacea to Enhanced Transferability Estimation](https://mlsanity.com/paper/2502.16471/) (2025)
- [How NOT to benchmark your SITE metric: Beyond Static Leaderboards and Towards Realistic Evaluation](https://mlsanity.com/paper/2510.06448/) (2025)
- [Selective Cross-Task Distillation](https://mlsanity.com/paper/2204.11526/) (2022)
- [To transfer or not transfer: Unified transferability metric and analysis](https://mlsanity.com/paper/2305.07741/) (2023)

## Abstract (arXiv)

Transfer learning has become a popular method for leveraging pre-trained models in computer vision. However, without performing computationally expensive fine-tuning, it is difficult to quantify which pre-trained source models are suitable for a specific target task, or, conversely, to which tasks a pre-trained source model can be easily adapted to. In this work, we propose Gaussian Bhattacharyya Coefficient (GBC), a novel method for quantifying transferability between a source model and a target dataset. In a first step we embed all target images in the feature space defined by the source model, and represent them with per-class Gaussians. Then, we estimate their pairwise class separability using the Bhattacharyya coefficient, yielding a simple and effective measure of how well the source model transfers to the target task. We evaluate GBC on image classification tasks in the context of dataset and architecture selection. Further, we also perform experiments on the more complex semantic segmentation transferability estimation task. We demonstrate that GBC outperforms state-of-the-art transferability metrics on most evaluation criteria in the semantic segmentation settings, matches the performance of top methods for dataset transferability in image classification, and performs best on architecture selection problems for image classification.