# Few-Shot Video Recognition via Hierarchical Metric Learning

arXiv: https://arxiv.org/abs/2609.05242 · HTML: https://mlsanity.com/paper/2609.05242/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 4 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [Few-Shot Video Classification via Temporal Alignment](https://mlsanity.com/paper/1906.11415/) (2019)
- [Temporal-Relational CrossTransformers for Few-Shot Action Recognition](https://mlsanity.com/paper/2101.06184/) (2021)
- [TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognition](https://mlsanity.com/paper/1907.09021/) (2019)
- [Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition](https://mlsanity.com/paper/2010.09982/) (2020)
- [Hybrid Relation Guided Set Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2204.13423/) (2022)
- [ProtoGAN: Towards Few Shot Learning for Action Recognition](https://mlsanity.com/paper/1909.07945/) (2019)
- [CLIP-guided Prototype Modulating for Few-shot Action Recognition](https://mlsanity.com/paper/2303.02982/) (2023)
- [MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition](https://mlsanity.com/paper/2304.00946/) (2023)
- [TA2N: Two-Stage Action Alignment Network for Few-shot Action Recognition](https://mlsanity.com/paper/2107.04782/) (2021)
- [MVP-Shot: Multi-Velocity Progressive-Alignment Framework for Few-Shot Action Recognition](https://mlsanity.com/paper/2405.02077/) (2024)
- [Boosting Few-shot Action Recognition with Graph-guided Hybrid Matching](https://mlsanity.com/paper/2308.09346/) (2023)
- [On the Importance of Spatial Relations for Few-shot Action Recognition](https://mlsanity.com/paper/2308.07119/) (2023)
- [Frame Order Matters: A Temporal Sequence-Aware Model for Few-Shot Action Recognition](https://mlsanity.com/paper/2408.12475/) (2024)
- [Temporal Alignment-Free Video Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2504.05956/) (2025)
- [Multi-level Metric Learning for Few-shot Image Recognition](https://mlsanity.com/paper/2103.11383/) (2021)
- [Metric-Based Few-Shot Learning for Video Action Recognition](https://mlsanity.com/paper/1909.09602/) (2019)
- [Storyboard guided Alignment for Fine-grained Video Action Recognition](https://mlsanity.com/paper/2410.14238/) (2024)
- [Spatio-temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition](https://mlsanity.com/paper/2602.18043/) (2026)
- [LPN: Language-guided Prototypical Network for few-shot classification](https://mlsanity.com/paper/2307.01515/) (2023)
- [MPA: Multimodal Prototype Augmentation for Few-Shot Learning](https://mlsanity.com/paper/2602.10143/) (2026)

## Abstract (arXiv)

Few-shot action recognition (FSAR) aims to recognize unseen action categories with only a small number of annotated video samples. Recent works typically apply single-prototype supervision at the network output and fail to sufficiently exploit rich cross-frame global spatial information in videos. Even existing multi-level metric schemes only impose parallel prototype constraints on intermediate layers, without progressive supervision along the full feature pipeline, which results in limited generalization ability of the learned class prototypes. Inspired by this, we present a novel method, hierarchical metric learning for few-shot action recognition (HML-FSAR). First, a spatial-enhanced module is developed to capture cross-frame global spatial representations. Combined with temporal MHA, heterogeneous alignment, spatial-temporal feature fusion and dictionary learning modules, it constructs the complete feature processing pipeline. Second, a hierarchical metric learning (HML) strategy is embedded into HML-FSAR. Composed of center metric, alignment metric, contrastive metric, dictionary metric and prototype metric, HML imposes progressive multi-stage complementary constraints from frame-level representations to final class prototypes, so as to jointly optimize feature compactness, heterogeneous spatial-temporal alignment, inter-class discriminability and anti-noise robustness. The proposed HML-FSAR method is validated on five widely-used FSAR datasets, and experimental results fully demonstrate its effectiveness.