# TA2N: Two-Stage Action Alignment Network for Few-shot Action Recognition

arXiv: https://arxiv.org/abs/2107.04782 · HTML: https://mlsanity.com/paper/2107.04782/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 10 Jul 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Low-Shot Learning from Imaginary Data](https://mlsanity.com/paper/1801.05401/) (2018)
- [Few-Shot Video Classification via Temporal Alignment](https://mlsanity.com/paper/1906.11415/) (2019)
- [TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognition](https://mlsanity.com/paper/1907.09021/) (2019)
- [TAEN: Temporal Aware Embedding Network for Few-Shot Action Recognition](https://mlsanity.com/paper/2004.10141/) (2020)

## Built on by (most cited first, 30 of 35)

- [Implicit Temporal Modeling with Learnable Alignment for Video Recognition](https://mlsanity.com/paper/2304.10465/) (2023)
- [CLIP-guided Prototype Modulating for Few-shot Action Recognition](https://mlsanity.com/paper/2303.02982/) (2023)
- [MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition](https://mlsanity.com/paper/2304.00946/) (2023)
- [Compound Prototype Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2207.05515/) (2022)
- [M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition](https://mlsanity.com/paper/2308.03063/) (2023)
- [MVP-Shot: Multi-Velocity Progressive-Alignment Framework for Few-Shot Action Recognition](https://mlsanity.com/paper/2405.02077/) (2024)
- [Boosting Few-shot Action Recognition with Graph-guided Hybrid Matching](https://mlsanity.com/paper/2308.09346/) (2023)
- [Revisiting the Spatial and Temporal Modeling for Few-shot Action Recognition](https://mlsanity.com/paper/2301.07944/) (2023)
- [HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2301.03330/) (2023)
- [A Comprehensive Review of Few-shot Action Recognition](https://mlsanity.com/paper/2407.14744/) (2024)
- [Hierarchical Compositional Representations for Few-shot Action Recognition](https://mlsanity.com/paper/2208.09424/) (2022)
- [Temporal Alignment-Free Video Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2504.05956/) (2025)
- [Task-adaptive Spatial-Temporal Video Sampler for Few-shot Action Recognition](https://mlsanity.com/paper/2207.09759/) (2022)
- [Few-shot Action Recognition with Captioning Foundation Models](https://mlsanity.com/paper/2310.10125/) (2023)
- [Task-Adapter: Task-specific Adaptation of Image Models for Few-shot Action Recognition](https://mlsanity.com/paper/2408.00249/) (2024)
- [Few-shot Action Recognition via Intra- and Inter-Video Information Maximization](https://mlsanity.com/paper/2305.06114/) (2023)
- [Semantic-aware Video Representation for Few-shot Action Recognition](https://mlsanity.com/paper/2311.06218/) (2023)
- [Consistency Prototype Module and Motion Compensation for Few-Shot Action Recognition (CLIP-CPM2C)](https://mlsanity.com/paper/2312.01083/) (2023)
- [Spatio-temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition](https://mlsanity.com/paper/2602.18043/) (2026)
- [Video alignment using unsupervised learning of local and global features](https://mlsanity.com/paper/2304.06841/) (2023)
- [Task-Specific Alignment and Multiple Level Transformer for Few-Shot Action Recognition](https://mlsanity.com/paper/2307.01985/) (2023)
- [Sample Less, Learn More: Efficient Action Recognition via Frame Feature Restoration](https://mlsanity.com/paper/2307.14866/) (2023)
- [A survey on bias in machine learning research](https://mlsanity.com/paper/2308.11254/) (2023)
- [Multi-view Distillation based on Multi-modal Fusion for Few-shot Action Recognition(CLIP-M2DF)](https://mlsanity.com/paper/2401.08345/) (2024)
- [Joint Image-Instance Spatial-Temporal Attention for Few-shot Action Recognition](https://mlsanity.com/paper/2503.14430/) (2025)
- [TimePoint: Accelerated Time Series Alignment via Self-Supervised Keypoint and Descriptor Learning](https://mlsanity.com/paper/2505.23475/) (2025)
- [D2ST-Adapter: Disentangled-and-Deformable Spatio-Temporal Adapter for Few-shot Action Recognition](https://mlsanity.com/paper/2312.01431/) (2023)
- [SOAP: Enhancing Spatio-Temporal Relation and Motion Information Capturing for Few-Shot Action Recognition](https://mlsanity.com/paper/2407.16344/) (2024)
- [Rethinking matching-based few-shot action recognition](https://mlsanity.com/paper/2303.16084/) (2023)
- [Video-to-Task Learning via Motion-Guided Attention for Few-Shot Action Recognition](https://mlsanity.com/paper/2411.11335/) (2024)

## Abstract (arXiv)

Few-shot action recognition aims to recognize novel action classes (query) using just a few samples (support). The majority of current approaches follow the metric learning paradigm, which learns to compare the similarity between videos. Recently, it has been observed that directly measuring this similarity is not ideal since different action instances may show distinctive temporal distribution, resulting in severe misalignment issues across query and support videos. In this paper, we arrest this problem from two distinct aspects -- action duration misalignment and action evolution misalignment. We address them sequentially through a Two-stage Action Alignment Network (TA2N). The first stage locates the action by learning a temporal affine transform, which warps each video feature to its action duration while dismissing the action-irrelevant feature (e.g. background). Next, the second stage coordinates query feature to match the spatial-temporal action evolution of support by performing temporally rearrange and spatially offset prediction. Extensive experiments on benchmark datasets show the potential of the proposed method in achieving state-of-the-art performance for few-shot action recognition.The code of this project can be found at https://github.com/R00Kie-Liu/TA2N