# Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learning

arXiv: https://arxiv.org/abs/2108.06647 · HTML: https://mlsanity.com/paper/2108.06647/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 15 Aug 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding](https://mlsanity.com/paper/2004.06704/) (2020)
- [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020)
- [Few-Shot Video Classification via Temporal Alignment](https://mlsanity.com/paper/1906.11415/) (2019)
- [Temporal-Relational CrossTransformers for Few-Shot Action Recognition](https://mlsanity.com/paper/2101.06184/) (2021)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [Attentional Pooling for Action Recognition](https://mlsanity.com/paper/1711.01467/) (2017)
- [Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition](https://mlsanity.com/paper/2010.09982/) (2020)
- [Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data](https://mlsanity.com/paper/1502.06648/) (2015)
- [HAA500: Human-Centric Atomic Action Dataset with Curated Videos](https://mlsanity.com/paper/2009.05224/) (2020)
- [Few-shot Action Recognition with Prototype-centered Attentive Learning](https://mlsanity.com/paper/2101.08085/) (2021)
- [Fine-grained Activity Recognition in Baseball Videos](https://mlsanity.com/paper/1804.03247/) (2018)
- [Cooperative Bi-path Metric for Few-shot Learning](https://mlsanity.com/paper/2008.04031/) (2020)
- [Fine-grained Video Categorization with Redundancy Reduction Attention](https://mlsanity.com/paper/1810.11189/) (2018)
- [Compositional Few-Shot Recognition with Primitive Discovery and Enhancing](https://mlsanity.com/paper/2005.06047/) (2020)

## Built on by (most cited first, 7 of 7)

- [A Comprehensive Review of Few-shot Action Recognition](https://mlsanity.com/paper/2407.14744/) (2024)
- [Learning Cross-Image Object Semantic Relation in Transformer for Few-Shot Fine-Grained Image Classification](https://mlsanity.com/paper/2207.00784/) (2022)
- [Learning Causal Domain-Invariant Temporal Dynamics for Few-Shot Action Recognition](https://mlsanity.com/paper/2402.12706/) (2024)
- [Exploring Effective Knowledge Transfer for Few-shot Object Detection](https://mlsanity.com/paper/2210.02021/) (2022)
- [ActAlign: Zero-Shot Fine-Grained Video Classification via Language-Guided Sequence Alignment](https://mlsanity.com/paper/2506.22967/) (2025)
- [Joint Image-Instance Spatial-Temporal Attention for Few-shot Action Recognition](https://mlsanity.com/paper/2503.14430/) (2025)
- [A2M2-Net: Adaptively Aligned Multi-Scale Moment for Few-Shot Action Recognition](https://mlsanity.com/paper/2509.17638/) (2025)

## Abstract (arXiv)

Fine-grained action recognition is attracting increasing attention due to the emerging demand of specific action understanding in real-world applications, whereas the data of rare fine-grained categories is very limited. Therefore, we propose the few-shot fine-grained action recognition problem, aiming to recognize novel fine-grained actions with only few samples given for each class. Although progress has been made in coarse-grained actions, existing few-shot recognition methods encounter two issues handling fine-grained actions: the inability to capture subtle action details and the inadequacy in learning from data with low inter-class variance. To tackle the first issue, a human vision inspired bidirectional attention module (BAM) is proposed. Combining top-down task-driven signals with bottom-up salient stimuli, BAM captures subtle action details by accurately highlighting informative spatio-temporal regions. To address the second issue, we introduce contrastive meta-learning (CML). Compared with the widely adopted ProtoNet-based method, CML generates more discriminative video representations for low inter-class variance data, since it makes full use of potential contrastive pairs in each training episode. Furthermore, to fairly compare different models, we establish specific benchmark protocols on two large-scale fine-grained action recognition datasets. Extensive experiments show that our method consistently achieves state-of-the-art performance across evaluated tasks.