# Hierarchical Variational Memory for Few-shot Learning Across Domains

arXiv: https://arxiv.org/abs/2112.08181 · HTML: https://mlsanity.com/paper/2112.08181/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 60 references that are in the map.
- First posted on arXiv 15 Dec 2021.

## Builds on (most cited first, 30 of 44)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Meta-learning with differentiable closed-form solvers](https://mlsanity.com/paper/1805.08136/) (2018)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Using Deep Learning for Image-Based Plant Disease Detection](https://mlsanity.com/paper/1604.03169/) (2016)
- [Cross Attention Network for Few-shot Classification](https://mlsanity.com/paper/1910.07677/) (2019)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation](https://mlsanity.com/paper/2001.08735/) (2020)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Neural Episodic Control](https://mlsanity.com/paper/1703.01988/) (2017)
- [Learning to Learn with Variational Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2007.07645/) (2020)
- [Finding Task-Relevant Features for Few-Shot Learning by Category Traversal](https://mlsanity.com/paper/1905.11116/) (2019)
- [Infinite Mixture Prototypes for Few-Shot Learning](https://mlsanity.com/paper/1902.04552/) (2019)

## Built on by (most cited first, 14 of 14)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [ProtoDiff: Learning to Learn Prototypical Networks by Task-Guided Diffusion](https://mlsanity.com/paper/2306.14770/) (2023)
- [Energy-Based Test Sample Adaptation for Domain Generalization](https://mlsanity.com/paper/2302.11215/) (2023)
- [Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey](https://mlsanity.com/paper/2303.08557/) (2023)
- [Exploring Cross-Domain Few-Shot Classification via Frequency-Aware Prompting](https://mlsanity.com/paper/2406.16422/) (2024)
- [Towards Few-Shot Learning in the Open World: A Review and Beyond](https://mlsanity.com/paper/2408.09722/) (2024)
- [Adaptive Parametric Prototype Learning for Cross-Domain Few-Shot Classification](https://mlsanity.com/paper/2309.01342/) (2023)
- [EMO: Episodic Memory Optimization for Few-Shot Meta-Learning](https://mlsanity.com/paper/2306.05189/) (2023)
- [One-Shot Learning of Visual Path Navigation for Autonomous Vehicles](https://mlsanity.com/paper/2306.08865/) (2023)
- [AdaTreeFormer: Few Shot Domain Adaptation for Tree Counting from a Single High-Resolution Image](https://mlsanity.com/paper/2402.02956/) (2024)
- [Association Graph Learning for Multi-Task Classification with Category Shifts](https://mlsanity.com/paper/2210.04637/) (2022)
- [Adaptive Weighted Co-Learning for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2312.03928/) (2023)
- [A Cross-Domain Few-Shot Learning Method Based on Domain Knowledge Mapping](https://mlsanity.com/paper/2504.06608/) (2025)
- [Variational Task Vector Composition](https://mlsanity.com/paper/2509.18208/) (2025)

## Abstract (arXiv)

Neural memory enables fast adaptation to new tasks with just a few training samples. Existing memory models store features only from the single last layer, which does not generalize well in presence of a domain shift between training and test distributions. Rather than relying on a flat memory, we propose a hierarchical alternative that stores features at different semantic levels. We introduce a hierarchical prototype model, where each level of the prototype fetches corresponding information from the hierarchical memory. The model is endowed with the ability to flexibly rely on features at different semantic levels if the domain shift circumstances so demand. We meta-learn the model by a newly derived hierarchical variational inference framework, where hierarchical memory and prototypes are jointly optimized. To explore and exploit the importance of different semantic levels, we further propose to learn the weights associated with the prototype at each level in a data-driven way, which enables the model to adaptively choose the most generalizable features. We conduct thorough ablation studies to demonstrate the effectiveness of each component in our model. The new state-of-the-art performance on cross-domain and competitive performance on traditional few-shot classification further substantiates the benefit of hierarchical variational memory.