mlsanityData to 7 Oct 2026

Universal Representation Learning from Multiple Domains for Few-shot Classification

arXiv 2103.13841 ↗First posted 25 Mar 2021cs.CVcited by 56

Not on the map. Why?

Abstract (arXiv)

In this paper, we look at the problem of few-shot classification that aims to learn a classifier for previously unseen classes and domains from few labeled samples. Recent methods use adaptation networks for aligning their features to new domains or select the relevant features from multiple domain-specific feature extractors. In this work, we propose to learn a single set of universal deep representations by distilling knowledge of multiple separately trained networks after co-aligning their features with the help of adapters and centered kernel alignment. We show that the universal representations can be further refined for previously unseen domains by an efficient adaptation step in a similar spirit to distance learning methods. We rigorously evaluate our model in the recent Meta-Dataset benchmark and demonstrate that it significantly outperforms the previous methods while being more efficient. Our code will be available at https://github.com/VICO-UoE/URL.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Builds on 41 of its 57 references that are in the map. First posted on arXiv 25 Mar 2021.

Builds on

41 of 57 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  8. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2013Describing Textures in the Wildcited by 2,110
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
29 more
  1. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  2. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  4. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  5. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  6. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  7. 2019Contrastive Representation Distillationcited by 734
  8. 2019A Closer Look at Few-shot Classificationcited by 679
  9. 2017Learning multiple visual domains with residual adapterscited by 553
  10. 2018Born Again Neural Networkscited by 539
  11. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  12. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  13. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  14. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  15. 2018How to train your MAMLcited by 306
  16. 2019A Baseline for Few-Shot Image Classificationcited by 298
  17. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  18. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  19. 2018Efficient parametrization of multi-domain deep neural networkscited by 168
  20. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  21. 2019Improved Few-Shot Visual Classificationcited by 114
  22. 2019BAM! Born-Again Multi-Task Networks for Natural Language Understandingcited by 85
  23. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  24. 2020Knowledge Distillation for Multi-task Learningcited by 51
  25. 2020TaskNorm: Rethinking Batch Normalization for Meta-Learningcited by 50
  26. 2017Universal representations:The missing link between faces, text, planktons, and cat breedscited by 45
  27. 2019Learning Metrics from Teachers: Compact Networks for Image Embeddingcited by 38
  28. 2020Optimized Generic Feature Learning for Few-shot Classification across Domainscited by 23
  29. 2019Graph Representation Learning via Multi-task Knowledge Distillationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 16; 2023: 16; 2024: 13; 2025: 7; 2026: 1.

56 papers.

Sort:

By area:

  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  2. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
  3. 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 123
  4. 2022Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalizationcited by 97
  5. 2022Head2Toe: Utilizing Intermediate Representations for Better Transfer Learningcited by 83
  6. 2021Cross-domain Few-shot Learning with Task-specific Adapterscited by 79
  7. 2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
  8. 2023A Closer Look at Few-shot Classification Againcited by 46
  9. 2023Meta-Album: Multi-domain Meta-Dataset for Few-Shot Image Classificationcited by 29
  10. 2021Deep Metric Learning for Few-Shot Image Classification: A Review of Recent Developmentscited by 23
  11. 2022Universal Representations: A Unified Look at Multiple Task and Domain Learningcited by 23
  12. 2021Forward Compatible Training for Large-Scale Embedding Retrieval Systemscited by 19
38 more
  1. 2023Exploring Efficient Few-shot Adaptation for Vision Transformerscited by 16
  2. 2025Reliable Few-shot Learning under Dual Noisescited by 16
  3. 2022Learning useful representations for shifting tasks and distributionscited by 14
  4. 2023Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Surveycited by 14
  5. 2022Transferability-Guided Cross-Domain Cross-Task Transfer Learningcited by 13
  6. 2023DETA: Denoised Task Adaptation for Few-Shot Learningcited by 13
  7. 2024A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Modelscited by 13
  8. 2025Empirical Analysis of Sim-and-Real Cotraining of Diffusion Policies for Planar Pushing from Pixelscited by 12
  9. 2023Meta-Learning with a Geometry-Adaptive Preconditionercited by 11
  10. 2024Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learningcited by 11
  11. 2023Learning to Drive Anywherecited by 9
  12. 2022Few-Shot Learning by Dimensionality Reduction in Gradient Spacecited by 8
  13. 2022Few-shot Metric Learning: Online Adaptation of Embedding for Retrievalcited by 8
  14. 2024HumanVLM: Foundation for Human-Scene Vision-Language Modelcited by 8
  15. 2024Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learningcited by 7
  16. 2023Is Pre-training Truly Better Than Meta-Learning?cited by 6
  17. 2024Towards Few-Shot Learning in the Open World: A Review and Beyondcited by 5
  18. 2022Cross-Domain Few-Shot Classification via Inter-Source Stylizationcited by 4
  19. 2022Schedule-Robust Online Continual Learningcited by 4
  20. 2023Meta Omnium: A Benchmark for General-Purpose Learning-to-Learncited by 4
  21. 2023DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learningcited by 4
  22. 2023Neural Fine-Tuning Search for Few-Shot Learningcited by 4
  23. 2024MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependencecited by 4
  24. 2025A Theoretical Framework for OOD Robustness in Transformers using Gevrey Classescited by 4
  25. 2022NeurIPS'22 Cross-Domain MetaDL competition: Design and baseline resultscited by 3
  26. 2023A Survey of Deep Visual Cross-Domain Few-Shot Learningcited by 3
  27. 2023Leveraging Normalization Layer in Adapters With Progressive Learning and Adaptive Distillation for Cross-Domain Few-Shot Learningcited by 3
  28. 2024Fine-tuning with Very Large Dropoutcited by 3
  29. 2024OVMR: Open-Vocabulary Recognition with Multi-Modal Referencescited by 3
  30. 2024Towards Model-Agnostic Dataset Condensation by Heterogeneous Modelscited by 3
  31. 2024Mind the Gap Between Prototypes and Images in Cross-domain Finetuningcited by 3
  32. 2024Context-Aware Multimodal Pretrainingcited by 3
  33. 2022Feature Extractor Stacking for Cross-domain Few-shot Learningcited by 2
  34. 2022Robust Meta-Representation Learning via Global Label Inference and Classificationcited by 2
  35. 2024Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysiscited by 2
  36. 2023AI-based association analysis for medical imaging using latent-space geometric confounder correctioncited by 1
  37. 2025FAD: Frequency Adaptation and Diversion for Cross-domain Few-shot Learningcited by 1
  38. 2022UV R-CNN: Stable and Efficient Dense Human Pose Estimationcited by 0

50 most cited of 56 All 56 (JSON)