# Human Action Sequence Classification

arXiv: https://arxiv.org/abs/1910.02602 · HTML: https://mlsanity.com/paper/1910.02602/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2019.

## Built on by (most cited first, 3 of 3)

- [With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition](https://mlsanity.com/paper/2111.01024/) (2021)
- [TIM: A Time Interval Machine for Audio-Visual Action Recognition](https://mlsanity.com/paper/2404.05559/) (2024)
- [OWL (Observe, Watch, Listen): Audiovisual Temporal Context for Localizing Actions in Egocentric Videos](https://mlsanity.com/paper/2202.04947/) (2022)

## Abstract (arXiv)

This paper classifies human action sequences from videos using a machine translation model. In contrast to classical human action classification which outputs a set of actions, our method output a sequence of action in the chronological order of the actions performed by the human. Therefore our method is evaluated using sequential performance measures such as Bilingual Evaluation Understudy (BLEU) scores. Action sequence classification has many applications such as learning from demonstration, action segmentation, detection, localization and video captioning. Furthermore, we use our model that is trained to output action sequences to solve downstream tasks; such as video captioning and action localization. We obtain state of the art results for video captioning in challenging Charades dataset obtaining BLEU-4 score of 34.8 and METEOR score of 33.6 outperforming previous state-of-the-art of 18.8 and 19.5 respectively. Similarly, on ActivityNet captioning, we obtain excellent results in-terms of ROUGE (20.24) and CIDER (37.58) scores. For action localization, without using any explicit start/end action annotations, our method obtains localization performance of 22.2 mAP outperforming prior fully supervised methods.