# Few-shot Action Recognition with Prototype-centered Attentive Learning

arXiv: https://arxiv.org/abs/2101.08085 · HTML: https://mlsanity.com/paper/2101.08085/

## Key facts

- Cited by 28 papers in the map (data through 13 Oct 2026).
- Builds on 35 of its 44 references that are in the map.
- First posted on arXiv 20 Jan 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [A Closer Look at Few-shot Classification](https://mlsanity.com/paper/1904.04232/) (2019)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions](https://mlsanity.com/paper/1812.03664/) (2018)
- [CrossTransformers: spatially-aware few-shot transfer](https://mlsanity.com/paper/2007.11498/) (2020)
- [Disentangled Non-Local Neural Networks](https://mlsanity.com/paper/2006.06668/) (2020)
- [Negative Margin Matters: Understanding Margin in Few-shot Classification](https://mlsanity.com/paper/2003.12060/) (2020)
- [Few-Shot Video Classification via Temporal Alignment](https://mlsanity.com/paper/1906.11415/) (2019)
- [Incremental Few-Shot Object Detection](https://mlsanity.com/paper/2003.04668/) (2020)
- [Finding Task-Relevant Features for Few-Shot Learning by Category Traversal](https://mlsanity.com/paper/1905.11116/) (2019)

## Built on by (most cited first, 28 of 28)

- [Prompting Visual-Language Models for Efficient Video Understanding](https://mlsanity.com/paper/2112.04478/) (2021)
- [CLIP-guided Prototype Modulating for Few-shot Action Recognition](https://mlsanity.com/paper/2303.02982/) (2023)
- [Compound Prototype Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2207.05515/) (2022)
- [M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition](https://mlsanity.com/paper/2308.03063/) (2023)
- [Proposal-Free Temporal Action Detection via Global Segmentation Mask Learning](https://mlsanity.com/paper/2207.06580/) (2022)
- [MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognition](https://mlsanity.com/paper/2305.07214/) (2023)
- [Use Your Head: Improving Long-Tail Video Recognition](https://mlsanity.com/paper/2304.01143/) (2023)
- [Prototypical Calibrating Ambiguous Samples for Micro-Action Recognition](https://mlsanity.com/paper/2412.14719/) (2024)
- [A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmark](https://mlsanity.com/paper/2110.12358/) (2021)
- [Revisiting the Spatial and Temporal Modeling for Few-shot Action Recognition](https://mlsanity.com/paper/2301.07944/) (2023)
- [Multimodal Prototype-Enhanced Network for Few-Shot Action Recognition](https://mlsanity.com/paper/2212.04873/) (2022)
- [Few-Shot Action Recognition with Compromised Metric via Optimal Transport](https://mlsanity.com/paper/2104.03737/) (2021)
- [A Comprehensive Review of Few-shot Action Recognition](https://mlsanity.com/paper/2407.14744/) (2024)
- [MASTAF: A Model-Agnostic Spatio-Temporal Attention Fusion Network for Few-shot Video Classification](https://mlsanity.com/paper/2112.04585/) (2021)
- [Task-Adapter: Task-specific Adaptation of Image Models for Few-shot Action Recognition](https://mlsanity.com/paper/2408.00249/) (2024)
- [Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learning](https://mlsanity.com/paper/2108.06647/) (2021)
- [Augmenting and Aligning Snippets for Few-Shot Video Domain Adaptation](https://mlsanity.com/paper/2303.10451/) (2023)
- [Learning Causal Domain-Invariant Temporal Dynamics for Few-Shot Action Recognition](https://mlsanity.com/paper/2402.12706/) (2024)
- [Anomaly Crossing: New Horizons for Video Anomaly Detection as Cross-domain Few-shot Learning](https://mlsanity.com/paper/2112.06320/) (2021)
- [VLG: General Video Recognition with Web Textual Knowledge](https://mlsanity.com/paper/2212.01638/) (2022)
- [Task-Specific Alignment and Multiple Level Transformer for Few-Shot Action Recognition](https://mlsanity.com/paper/2307.01985/) (2023)
- [Multi-view Distillation based on Multi-modal Fusion for Few-shot Action Recognition(CLIP-M2DF)](https://mlsanity.com/paper/2401.08345/) (2024)
- [Joint Image-Instance Spatial-Temporal Attention for Few-shot Action Recognition](https://mlsanity.com/paper/2503.14430/) (2025)
- [TNT: Text-Conditioned Network with Transductive Inference for Few-Shot Video Classification](https://mlsanity.com/paper/2106.11173/) (2021)
- [Rethinking matching-based few-shot action recognition](https://mlsanity.com/paper/2303.16084/) (2023)
- [Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work](https://mlsanity.com/paper/2306.01929/) (2023)
- [Task-Adapter++: Task-specific Adaptation with Order-aware Alignment for Few-shot Action Recognition](https://mlsanity.com/paper/2505.06002/) (2025)
- [A2M2-Net: Adaptively Aligned Multi-Scale Moment for Few-Shot Action Recognition](https://mlsanity.com/paper/2509.17638/) (2025)

## Abstract (arXiv)

Few-shot action recognition aims to recognize action classes with few training samples. Most existing methods adopt a meta-learning approach with episodic training. In each episode, the few samples in a meta-training task are split into support and query sets. The former is used to build a classifier, which is then evaluated on the latter using a query-centered loss for model updating. There are however two major limitations: lack of data efficiency due to the query-centered only loss design and inability to deal with the support set outlying samples and inter-class distribution overlapping problems. In this paper, we overcome both limitations by proposing a new Prototype-centered Attentive Learning (PAL) model composed of two novel components. First, a prototype-centered contrastive learning loss is introduced to complement the conventional query-centered learning objective, in order to make full use of the limited training samples in each episode. Second, PAL further integrates a hybrid attentive learning mechanism that can minimize the negative impacts of outliers and promote class separation. Extensive experiments on four standard few-shot action benchmarks show that our method clearly outperforms previous state-of-the-art methods, with the improvement particularly significant (10+%) on the most challenging fine-grained action recognition benchmark.