# Adapted Deep Embeddings: A Synthesis of Methods for k-Shot Inductive Transfer Learning

arXiv: https://arxiv.org/abs/1805.08402 · HTML: https://mlsanity.com/paper/1805.08402/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 17 of 17)

- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [Self-Adversarial Disentangling for Specific Domain Adaptation](https://mlsanity.com/paper/2108.03553/) (2021)
- [Learning to Defer with Limited Expert Predictions](https://mlsanity.com/paper/2304.07306/) (2023)
- [Few-Shot Learning with a Strong Teacher](https://mlsanity.com/paper/2107.00197/) (2021)
- [EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients](https://mlsanity.com/paper/2201.09699/) (2022)
- [Few-Shot Adaptation of Pre-Trained Networks for Domain Shift](https://mlsanity.com/paper/2205.15234/) (2022)
- [Deep transfer learning for image classification: a survey](https://mlsanity.com/paper/2205.09904/) (2022)
- [Breaking Immutable: Information-Coupled Prototype Elaboration for Few-Shot Object Detection](https://mlsanity.com/paper/2211.14782/) (2022)
- [Transfer learning based few-shot classification using optimal transport mapping from preprocessed latent space of backbone neural network](https://mlsanity.com/paper/2102.05176/) (2021)
- [Contextualizing Meta-Learning via Learning to Decompose](https://mlsanity.com/paper/2106.08112/) (2021)
- [Practical Continual Forgetting for Pre-trained Vision Models](https://mlsanity.com/paper/2501.09705/) (2025)
- [Episodic fine-tuning prototypical networks for optimization-based few-shot learning: Application to audio classification](https://mlsanity.com/paper/2410.05302/) (2024)
- [ProtoShotXAI: Using Prototypical Few-Shot Architecture for Explainable AI](https://mlsanity.com/paper/2110.11597/) (2021)
- [PEFSL: A deployment Pipeline for Embedded Few-Shot Learning on a FPGA SoC](https://mlsanity.com/paper/2404.19354/) (2024)
- [UNEM: UNrolled Generalized EM for Transductive Few-Shot Learning](https://mlsanity.com/paper/2412.16739/) (2024)
- [A Maximum Log-Likelihood Method for Imbalanced Few-Shot Learning Tasks](https://mlsanity.com/paper/2211.14668/) (2022)
- [Episode-specific Fine-tuning for Metric-based Few-shot Learners with Optimization-based Training](https://mlsanity.com/paper/2506.17499/) (2025)

## Abstract (arXiv)

The focus in machine learning has branched beyond training classifiers on a single task to investigating how previously acquired knowledge in a source domain can be leveraged to facilitate learning in a related target domain, known as inductive transfer learning. Three active lines of research have independently explored transfer learning using neural networks. In weight transfer, a model trained on the source domain is used as an initialization point for a network to be trained on the target domain. In deep metric learning, the source domain is used to construct an embedding that captures class structure in both the source and target domains. In few-shot learning, the focus is on generalizing well in the target domain based on a limited number of labeled examples. We compare state-of-the-art methods from these three paradigms and also explore hybrid adapted-embedding methods that use limited target-domain data to fine tune embeddings constructed from source-domain data. We conduct a systematic comparison of methods in a variety of domains, varying the number of labeled instances available in the target domain (k), as well as the number of target-domain classes. We reach three principal conclusions: (1) Deep embeddings are far superior, compared to weight transfer, as a starting point for inter-domain transfer or model re-use (2) Our hybrid methods robustly outperform every few-shot learning and every deep metric learning method previously proposed, with a mean error reduction of 34% over state-of-the-art. (3) Among loss functions for discovering embeddings, the histogram loss (Ustinova & Lempitsky, 2016) is most robust. We hope our results will motivate a unification of research in weight transfer, deep metric learning, and few-shot learning.