# A Generative Approach to Zero-Shot and Few-Shot Action Recognition

arXiv: https://arxiv.org/abs/1801.09086 · HTML: https://mlsanity.com/paper/1801.09086/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2018.

## Built on by (most cited first, 30 of 39)

- [Revisiting Classifier: Transferring Vision-Language Models for Video Recognition](https://mlsanity.com/paper/2207.01297/) (2022)
- [Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimization](https://mlsanity.com/paper/2302.00624/) (2023)
- [Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models](https://mlsanity.com/paper/2301.00182/) (2022)
- [Compound Prototype Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2207.05515/) (2022)
- [Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition](https://mlsanity.com/paper/2109.01305/) (2021)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [Learning Implicit Temporal Alignment for Few-shot Video Classification](https://mlsanity.com/paper/2105.04823/) (2021)
- [Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models](https://mlsanity.com/paper/2207.07646/) (2022)
- [Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition](https://mlsanity.com/paper/2210.16820/) (2022)
- [Use Your Head: Improving Long-Tail Video Recognition](https://mlsanity.com/paper/2304.01143/) (2023)
- [Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Data](https://mlsanity.com/paper/2310.05010/) (2023)
- [Object Priors for Classifying and Localizing Unseen Actions](https://mlsanity.com/paper/2104.04715/) (2021)
- [Cross-modal Representation Learning for Zero-shot Action Recognition](https://mlsanity.com/paper/2205.01657/) (2022)
- [Synthetic Sample Selection for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2304.02846/) (2023)
- [CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition](https://mlsanity.com/paper/2101.07042/) (2021)
- [Multi-level Second-order Few-shot Learning](https://mlsanity.com/paper/2201.05916/) (2022)
- [A Comprehensive Review of Few-shot Action Recognition](https://mlsanity.com/paper/2407.14744/) (2024)
- [3D Skeleton-based Few-shot Action Recognition with JEANIE is not so Na"ive](https://mlsanity.com/paper/2112.12668/) (2021)
- [Hierarchical Compositional Representations for Few-shot Action Recognition](https://mlsanity.com/paper/2208.09424/) (2022)
- [Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment](https://mlsanity.com/paper/2402.04599/) (2024)
- [Zero-Shot Aerial Object Detection with Visual Description Regularization](https://mlsanity.com/paper/2402.18233/) (2024)
- [Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2101.03292/) (2021)
- [Orthogonal Temporal Interpolation for Zero-Shot Video Recognition](https://mlsanity.com/paper/2308.06897/) (2023)
- [Few Shot Activity Recognition Using Variational Inference](https://mlsanity.com/paper/2108.08990/) (2021)
- [Instance-level Few-shot Learning with Class Hierarchy Mining](https://mlsanity.com/paper/2304.07459/) (2023)
- [Continual Learning Improves Zero-Shot Action Recognition](https://mlsanity.com/paper/2410.10497/) (2024)
- [ZS-SLR: Zero-Shot Sign Language Recognition from RGB-D Videos](https://mlsanity.com/paper/2108.10059/) (2021)
- [HAA4D: Few-Shot Human Atomic Action Recognition via 3D Spatio-Temporal Skeletal Alignment](https://mlsanity.com/paper/2202.07308/) (2022)
- [Going Deeper into Recognizing Actions in Dark Environments: A Comprehensive Benchmark Study](https://mlsanity.com/paper/2202.09545/) (2022)
- [Telling Stories for Common Sense Zero-Shot Action Recognition](https://mlsanity.com/paper/2309.17327/) (2023)

## Abstract (arXiv)

We present a generative framework for zero-shot action recognition where some of the possible action classes do not occur in the training data. Our approach is based on modeling each action class using a probability distribution whose parameters are functions of the attribute vector representing that action class. In particular, we assume that the distribution parameters for any action class in the visual space can be expressed as a linear combination of a set of basis vectors where the combination weights are given by the attributes of the action class. These basis vectors can be learned solely using labeled data from the known (i.e., previously seen) action classes, and can then be used to predict the parameters of the probability distributions of unseen action classes. We consider two settings: (1) Inductive setting, where we use only the labeled examples of the seen action classes to predict the unseen action class parameters; and (2) Transductive setting which further leverages unlabeled data from the unseen action classes. Our framework also naturally extends to few-shot action recognition where a few labeled examples from unseen classes are available. Our experiments on benchmark datasets (UCF101, HMDB51 and Olympic) show significant performance improvements as compared to various baselines, in both standard zero-shot (disjoint seen and unseen classes) and generalized zero-shot learning settings.