# RNN Fisher Vectors for Action Recognition and Image Annotation

arXiv: https://arxiv.org/abs/1512.03958 · HTML: https://mlsanity.com/paper/1512.03958/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2015.

## Built on by (most cited first, 11 of 11)

- [IIP-Transformer: Intra-Inter-Part Transformer for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2110.13385/) (2021)
- [Graph Contrastive Learning for Skeleton-based Action Recognition](https://mlsanity.com/paper/2301.10900/) (2023)
- [Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition](https://mlsanity.com/paper/2110.13008/) (2021)
- [Survey of Action Recognition, Spotting and Spatio-Temporal Localization in Soccer -- Current Trends and Research Perspectives](https://mlsanity.com/paper/2309.12067/) (2023)
- [Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval](https://mlsanity.com/paper/2104.04991/) (2021)
- [FSAR: Federated Skeleton-based Action Recognition with Adaptive Topology Structure and Knowledge Distillation](https://mlsanity.com/paper/2306.11046/) (2023)
- [BDIQA: A New Dataset for Video Question Answering to Explore Cognitive Reasoning through Theory of Mind](https://mlsanity.com/paper/2402.07402/) (2024)
- [Learning to embed semantic similarity for joint image-text retrieval](https://mlsanity.com/paper/2210.03838/) (2022)
- [Improving Video Violence Recognition with Human Interaction Learning on 3D Skeleton Point Clouds](https://mlsanity.com/paper/2308.13866/) (2023)
- [Human Action Recognition (HAR) Using Skeleton-based Spatial Temporal Relative Transformer Network: ST-RTR](https://mlsanity.com/paper/2410.23806/) (2024)
- [SHARDeg: A Benchmark for Skeletal Human Action Recognition in Degraded Scenarios](https://mlsanity.com/paper/2505.18048/) (2025)

## Abstract (arXiv)

Recurrent Neural Networks (RNNs) have had considerable success in classifying and predicting sequences. We demonstrate that RNNs can be effectively used in order to encode sequences and provide effective representations. The methodology we use is based on Fisher Vectors, where the RNNs are the generative probabilistic models and the partial derivatives are computed using backpropagation. State of the art results are obtained in two central but distant tasks, which both rely on sequences: video action recognition and image annotation. We also show a surprising transfer learning result from the task of image annotation to the task of video action recognition.