mlsanityData to 14 Oct 2026

Temporal-Relational CrossTransformers for Few-Shot Action Recognition

arXiv 2101.06184 ↗First posted 15 Jan 2021cs.CVcited by 84

Not on the map. Why?

Abstract (arXiv)

We propose a novel approach to few-shot action recognition, finding temporally-corresponding frame tuples between the query and videos in the support set. Distinct from previous few-shot works, we construct class prototypes using the CrossTransformer attention mechanism to observe relevant sub-sequences of all support videos, rather than using class averages or single best matches. Video representations are formed from ordered tuples of varying numbers of frames, which allows sub-sequences of actions at different speeds and temporal offsets to be compared. Our proposed Temporal-Relational CrossTransformers (TRX) achieve state-of-the-art results on few-shot splits of Kinetics, Something-Something V2 (SSv2), HMDB51 and UCF101. Importantly, our method outperforms prior work on SSv2 by a wide margin (12%) due to the its ability to model temporal relations. A detailed ablation showcases the importance of matching to multiple support set videos and learning higher-order relational CrossTransformers.

Key facts. Cited by 84 papers in the map (data through 14 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 15 Jan 2021.

Builds on

25 of 36 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2016Layer Normalizationcited by 5,669
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  7. 2016Matching Networks for One Shot Learningcited by 2,255
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  10. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  11. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  12. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
13 more
  1. 2017Temporal Relational Reasoning in Videoscited by 324
  2. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 315
  3. 2018Few-shot Object Detection via Feature Reweightingcited by 265
  4. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  5. 2019STM: SpatioTemporal and Motion Encoding for Action Recognitioncited by 142
  6. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  7. 2019Few-Shot Video Classification via Temporal Alignmentcited by 125
  8. 2019Improved Few-Shot Visual Classificationcited by 115
  9. 2019TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognitioncited by 71
  10. 2019Only Time Can Tell: Discovering Temporal Data for Temporal Modelingcited by 67
  11. 2019ProtoGAN: Towards Few Shot Learning for Action Recognitioncited by 41
  12. 2019Retro-Actions: Learning 'Close' by Time-Reversing 'Open' Videoscited by 9
  13. 2016Few-Shot Object Recognition from Machine-Labeled Web Imagescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 17; 2023: 21; 2024: 17; 2025: 15; 2026: 6.

84 papers.

Sort:

By area:

  1. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  2. 2022Video Transformers: A Surveycited by 63
  3. 2021Spatio-temporal Relation Modeling for Few-shot Action Recognitioncited by 53
  4. 2022Hybrid Relation Guided Set Matching for Few-shot Action Recognitioncited by 50
  5. 2023CLIP-guided Prototype Modulating for Few-shot Action Recognitioncited by 41
  6. 2023MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognitioncited by 37
  7. 2022Compound Prototype Matching for Few-shot Action Recognitioncited by 35
  8. 2023M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognitioncited by 34
  9. 2022DirecFormer: A Directed Attention in Transformer Approach to Robust Action Recognitioncited by 31
  10. 2022Proposal-Free Temporal Action Detection via Global Segmentation Mask Learningcited by 29
  11. 2021Few-shot Action Recognition with Prototype-centered Attentive Learningcited by 28
  12. 2022Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognitioncited by 27
38 more
  1. 2023What can a cook in Italy teach a mechanic in India? Action Recognition Generalisation Over Scenarios and Locationscited by 25
  2. 2024MVP-Shot: Multi-Velocity Progressive-Alignment Framework for Few-Shot Action Recognitioncited by 25
  3. 2022HyperShot: Few-Shot Learning by Kernel HyperNetworkscited by 24
  4. 2023Boosting Few-shot Action Recognition with Graph-guided Hybrid Matchingcited by 24
  5. 2022Label2Label: A Language Modeling Framework for Multi-Attribute Learningcited by 23
  6. 2023Use Your Head: Improving Long-Tail Video Recognitioncited by 22
  7. 2022Uncertainty-DTW for Time Series and Sequencescited by 21
  8. 2021A Closer Look at Few-Shot Video Classification: A New Baseline and Benchmarkcited by 16
  9. 2023Revisiting the Spatial and Temporal Modeling for Few-shot Action Recognitioncited by 15
  10. 2021A New Split for Evaluating True Zero-Shot Action Recognitioncited by 14
  11. 2022Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusionscited by 13
  12. 2022Semantic-aligned Fusion Transformer for One-shot Object Detectioncited by 13
  13. 2022Multimodal Prototype-Enhanced Network for Few-Shot Action Recognitioncited by 13
  14. 2023HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognitioncited by 12
  15. 2024Taylor Videos for Action Recognitioncited by 12
  16. 2023On the Importance of Spatial Relations for Few-shot Action Recognitioncited by 11
  17. 2024A Comprehensive Review of Few-shot Action Recognitioncited by 11
  18. 2022Hierarchical Compositional Representations for Few-shot Action Recognitioncited by 10
  19. 2024Trajectory-aligned Space-time Tokens for Few-shot Action Recognitioncited by 10
  20. 2024Frame Order Matters: A Temporal Sequence-Aware Model for Few-Shot Action Recognitioncited by 10
  21. 2025Temporal Alignment-Free Video Matching for Few-shot Action Recognitioncited by 10
  22. 2022Task-adaptive Spatial-Temporal Video Sampler for Few-shot Action Recognitioncited by 9
  23. 2023Few-shot Action Recognition with Captioning Foundation Modelscited by 9
  24. 2021MASTAF: A Model-Agnostic Spatio-Temporal Attention Fusion Network for Few-shot Video Classificationcited by 8
  25. 2024ActionHub: A Large-scale Action Video Description Dataset for Zero-shot Action Recognitioncited by 8
  26. 2024Task-Adapter: Task-specific Adaptation of Image Models for Few-shot Action Recognitioncited by 8
  27. 2021Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learningcited by 7
  28. 2023Augmenting and Aligning Snippets for Few-Shot Video Domain Adaptationcited by 7
  29. 2023Few-shot Action Recognition via Intra- and Inter-Video Information Maximizationcited by 7
  30. 2023Semantic-aware Video Representation for Few-shot Action Recognitioncited by 7
  31. 2024From CNNs to Transformers in Multimodal Human Action Recognition: A Surveycited by 7
  32. 2025Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognitioncited by 7
  33. 2022Knowledge Prompting for Few-shot Action Recognitioncited by 6
  34. 2025Uni-Hand: Universal Hand Motion Forecasting in Egocentric Viewscited by 5
  35. 2023MA-FSAR: Multimodal Adaptation of CLIP for Few-Shot Action Recognitioncited by 4
  36. 2023Telling Stories for Common Sense Zero-Shot Action Recognitioncited by 4
  37. 2023Consistency Prototype Module and Motion Compensation for Few-Shot Action Recognition (CLIP-CPM2C)cited by 4
  38. 2024Learning Causal Domain-Invariant Temporal Dynamics for Few-Shot Action Recognitioncited by 4

50 most cited of 84 All 84 (JSON)