End-to-End Semantic Video Transformer for Zero-Shot Action Recognition
Not on the map. Why?
Abstract (arXiv)
While video action recognition has been an active area of research for several years, zero-shot action recognition has only recently started gaining traction. In this work, we propose a novel end-to-end trained transformer model which is capable of capturing long range spatiotemporal dependencies efficiently, contrary to existing approaches which use 3D-CNNs. Moreover, to address a common ambiguity in the existing works about classes that can be considered as previously unseen, we propose a new experimentation setup that satisfies the zero-shot learning premise for action recognition by avoiding overlap between the training and testing classes. The proposed approach significantly outperforms the state of the arts in zero-shot action recognition in terms of the the top-1 accuracy on UCF-101, HMDB-51 and ActivityNet datasets. The code and proposed experimentation setup are available in GitHub: https://github.com/Secure-and-Intelligent-Systems-Lab/SemanticVideoTransformer
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 52 references that are in the map. First posted on arXiv 10 Mar 2022.
Builds on
34 of 52 references.
References · 34
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
- 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
- 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
- 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
- 2018A Short Note about Kinetics-600cited by 333
- 2017Feature Generating Networks for Zero-Shot Learningcited by 257
- 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
22 more
- 2019Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolutioncited by 140
- 2021VidTr: Video Transformer Without Convolutionscited by 112
- 2017Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Featurescited by 104
- 2018Cross-Modal and Hierarchical Modeling of Video and Textcited by 103
- 2020A Short Note on the Kinetics-700-2020 Human Action Datasetcited by 92
- 2020Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applicationscited by 80
- 2021Elaborative Rehearsal for Zero-shot Action Recognitioncited by 77
- 2019TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognitioncited by 71
- 2019Out-of-Distribution Detection for Generalized Zero-Shot Action Recognitioncited by 54
- 2016Learning Attributes Equals Multi-Source Domain Generalizationcited by 44
- 2018Towards Universal Representation for Unseen Action Recognitioncited by 44
- 2018A Generative Approach to Zero-Shot and Few-Shot Action Recognitioncited by 39
- 2019Spatiotemporal Pyramid Network for Video Action Recognitioncited by 37
- 2015Transductive Zero-Shot Action Recognition by Word-Vector Embeddingcited by 31
- 2016Zero-Shot Visual Recognition via Bidirectional Latent Embeddingcited by 27
- 2016Multi-Task Zero-Shot Action Recognition with Prioritised Data Augmentationcited by 27
- 2017Alternative Semantic Representations for Zero-Shot Human Action Recognitioncited by 27
- 2015Semantic Embedding Space for Zero-Shot Action Recognitioncited by 19
- 2019Action2Vec: A Crossmodal Embedding Approach to Action Learningcited by 19
- 2021A New Split for Evaluating True Zero-Shot Action Recognitioncited by 14
- 2021CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognitioncited by 12
- 2018Informed Democracy: Voting-based Novelty Detection for Action Recognitioncited by 11
Built on by
Citing papers per year: 2023: 1.
1 paper.