# Semantic Embedding Space for Zero-Shot Action Recognition

arXiv: https://arxiv.org/abs/1502.01540 · HTML: https://mlsanity.com/paper/1502.01540/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2015.

## Built on by (most cited first, 19 of 19)

- [Elaborative Rehearsal for Zero-shot Action Recognition](https://mlsanity.com/paper/2108.02833/) (2021)
- [Zero-Shot Temporal Action Detection via Vision-Language Prompting](https://mlsanity.com/paper/2207.08184/) (2022)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [Cross-modal Representation Learning for Zero-shot Action Recognition](https://mlsanity.com/paper/2205.01657/) (2022)
- [Towards Zero-shot Sign Language Recognition](https://mlsanity.com/paper/2201.05914/) (2022)
- [Zero-Shot Skeleton-based Action Recognition with Dual Visual-Text Alignment](https://mlsanity.com/paper/2409.14336/) (2024)
- [Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2101.03292/) (2021)
- [Zero-Shot Action Recognition from Diverse Object-Scene Compositions](https://mlsanity.com/paper/2110.13479/) (2021)
- [ActionHub: A Large-scale Action Video Description Dataset for Zero-shot Action Recognition](https://mlsanity.com/paper/2401.11654/) (2024)
- [Tell me what you see: A zero-shot action recognition method based on natural language descriptions](https://mlsanity.com/paper/2112.09976/) (2021)
- [Disentangled Action Recognition with Knowledge Bases](https://mlsanity.com/paper/2207.01708/) (2022)
- [ZS-SLR: Zero-Shot Sign Language Recognition from RGB-D Videos](https://mlsanity.com/paper/2108.10059/) (2021)
- [Learning Skill-Attributes for Transferable Assessment in Video](https://mlsanity.com/paper/2511.13993/) (2025)
- [Multi-Modal Zero-Shot Sign Language Recognition](https://mlsanity.com/paper/2109.00796/) (2021)
- [GAN for Vision, KG for Relation: a Two-stage Deep Network for Zero-shot Action Recognition](https://mlsanity.com/paper/2105.11789/) (2021)
- [End-to-End Semantic Video Transformer for Zero-Shot Action Recognition](https://mlsanity.com/paper/2203.05156/) (2022)
- [Learning Using Privileged Information for Zero-Shot Action Recognition](https://mlsanity.com/paper/2206.08632/) (2022)
- [Punching Bag vs. Punching Person: Motion Transferability in Videos](https://mlsanity.com/paper/2508.00085/) (2025)
- [Novel Semantic Prompting for Zero-Shot Action Recognition](https://mlsanity.com/paper/2603.08289/) (2026)

## Abstract (arXiv)

The number of categories for action recognition is growing rapidly. It is thus becoming increasingly hard to collect sufficient training data to learn conventional models for each category. This issue may be ameliorated by the increasingly popular 'zero-shot learning' (ZSL) paradigm. In this framework a mapping is constructed between visual features and a human interpretable semantic description of each category, allowing categories to be recognised in the absence of any training data. Existing ZSL studies focus primarily on image data, and attribute-based semantic representations. In this paper, we address zero-shot recognition in contemporary video action recognition tasks, using semantic word vector space as the common space to embed videos and category labels. This is more challenging because the mapping between the semantic space and space-time features of videos containing complex actions is more complex and harder to learn. We demonstrate that a simple self-training and data augmentation strategy can significantly improve the efficacy of this mapping. Experiments on human action datasets including HMDB51 and UCF101 demonstrate that our approach achieves the state-of-the-art zero-shot action recognition performance.