# Cross-Domain Few-Shot Learning by Representation Fusion

arXiv: https://arxiv.org/abs/2010.06498 · HTML: https://mlsanity.com/paper/2010.06498/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 30 of 32)

- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning](https://mlsanity.com/paper/2201.03529/) (2022)
- [Cross-domain Few-shot Learning with Task-specific Adapters](https://mlsanity.com/paper/2107.00358/) (2021)
- [Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data](https://mlsanity.com/paper/2107.11978/) (2021)
- [History Compression via Language Models in Reinforcement Learning](https://mlsanity.com/paper/2205.12258/) (2022)
- [A Dataset Perspective on Offline Reinforcement Learning](https://mlsanity.com/paper/2111.04714/) (2021)
- [Ranking Distance Calibration for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2112.00260/) (2021)
- [Understanding and Improving Transfer Learning of Deep Models via Neural Collapse](https://mlsanity.com/paper/2212.12206/) (2022)
- [Context-enriched molecule representations improve few-shot drug discovery](https://mlsanity.com/paper/2305.09481/) (2023)
- [Wave-SAN: Wavelet based Style Augmentation Network for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2203.07656/) (2022)
- [Semantic HELM: A Human-Readable Memory for Reinforcement Learning](https://mlsanity.com/paper/2306.09312/) (2023)
- [Hierarchical Variational Memory for Few-shot Learning Across Domains](https://mlsanity.com/paper/2112.08181/) (2021)
- [Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey](https://mlsanity.com/paper/2303.08557/) (2023)
- [Intermediate Layer Classifiers for OOD generalization](https://mlsanity.com/paper/2504.05461/) (2025)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learning](https://mlsanity.com/paper/2207.05742/) (2022)
- [Few-Shot Learning by Dimensionality Reduction in Gradient Space](https://mlsanity.com/paper/2206.03483/) (2022)
- [Improved Region Proposal Network for Enhanced Few-Shot Object Detection](https://mlsanity.com/paper/2308.07535/) (2023)
- [Cross-Domain Few-Shot Classification via Inter-Source Stylization](https://mlsanity.com/paper/2208.08015/) (2022)
- [Adaptive Parametric Prototype Learning for Cross-Domain Few-Shot Classification](https://mlsanity.com/paper/2309.01342/) (2023)
- [Anomaly Crossing: New Horizons for Video Anomaly Detection as Cross-domain Few-shot Learning](https://mlsanity.com/paper/2112.06320/) (2021)
- [Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification](https://mlsanity.com/paper/2311.02392/) (2023)
- [Rethinking the Use of Vision Transformers for AI-Generated Image Detection](https://mlsanity.com/paper/2512.04969/) (2025)
- [Improving the Generalization of Meta-learning on Unseen Domains via Adversarial Shift](https://mlsanity.com/paper/2107.11056/) (2021)
- [Selecting task with optimal transport self-supervised learning for few-shot classification](https://mlsanity.com/paper/2204.00289/) (2022)
- [PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks](https://mlsanity.com/paper/2403.11743/) (2024)
- [Adaptive Weighted Co-Learning for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2312.03928/) (2023)
- [A Cross-Domain Few-Shot Learning Method Based on Domain Knowledge Mapping](https://mlsanity.com/paper/2504.06608/) (2025)
- [AdaptMol: Adaptive Fusion from Sequence String to Topological Structure for Few-shot Drug Discovery](https://mlsanity.com/paper/2505.11878/) (2025)

## Abstract (arXiv)

In order to quickly adapt to new data, few-shot learning aims at learning from few examples, often by using already acquired knowledge. The new data often differs from the previously seen data due to a domain shift, that is, a change of the input-target distribution. While several methods perform well on small domain shifts like new target classes with similar inputs, larger domain shifts are still challenging. Large domain shifts may result in high-level concepts that are not shared between the original and the new domain, whereas low-level concepts like edges in images might still be shared and useful. For cross-domain few-shot learning, we suggest representation fusion to unify different abstraction levels of a deep neural network into one representation. We propose Cross-domain Hebbian Ensemble Few-shot learning (CHEF), which achieves representation fusion by an ensemble of Hebbian learners acting on different layers of a deep neural network. Ablation studies show that representation fusion is a decisive factor to boost cross-domain few-shot learning. On the few-shot datasets miniImagenet and tieredImagenet with small domain shifts, CHEF is competitive with state-of-the-art methods. On cross-domain few-shot benchmark challenges with larger domain shifts, CHEF establishes novel state-of-the-art results in all categories. We further apply CHEF on a real-world cross-domain application in drug discovery. We consider a domain shift from bioactive molecules to environmental chemicals and drugs with twelve associated toxicity prediction tasks. On these tasks, that are highly relevant for computational drug discovery, CHEF significantly outperforms all its competitors. Github: https://github.com/ml-jku/chef