# Video Action Understanding

arXiv: https://arxiv.org/abs/2010.06647 · HTML: https://mlsanity.com/paper/2010.06647/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 13 of 13)

- [VLTinT: Visual-Linguistic Transformer-in-Transformer for Coherent Video Paragraph Captioning](https://mlsanity.com/paper/2211.15103/) (2022)
- [ObjectMix: Data Augmentation by Copy-Pasting Objects in Videos for Action Recognition](https://mlsanity.com/paper/2204.00239/) (2022)
- [Uncertainty-boosted Robust Video Activity Anticipation](https://mlsanity.com/paper/2404.18648/) (2024)
- [O-MaMa: Learning Object Mask Matching between Egocentric and Exocentric Views](https://mlsanity.com/paper/2506.06026/) (2025)
- [Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos](https://mlsanity.com/paper/2309.05943/) (2023)
- [From Play to Replay: Composed Video Retrieval for Temporally Fine-Grained Videos](https://mlsanity.com/paper/2506.05274/) (2025)
- [Model-agnostic Multi-Domain Learning with Domain-Specific Adapters for Action Recognition](https://mlsanity.com/paper/2204.07270/) (2022)
- [Performance Evaluation of Action Recognition Models on Low Quality Videos](https://mlsanity.com/paper/2204.09166/) (2022)
- [S3Aug: Segmentation, Sampling, and Shift for Action Recognition](https://mlsanity.com/paper/2310.14556/) (2023)
- [VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstruction](https://mlsanity.com/paper/2511.18831/) (2025)
- [STROKEVISION-BENCH: A Multimodal Video And 2D Pose Benchmark For Tracking Stroke Recovery](https://mlsanity.com/paper/2509.07994/) (2025)
- [Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition](https://mlsanity.com/paper/2207.13306/) (2022)
- [MoExDA: Domain Adaptation for Edge-based Action Recognition](https://mlsanity.com/paper/2508.02981/) (2025)

## Abstract (arXiv)

Many believe that the successes of deep learning on image understanding problems can be replicated in the realm of video understanding. However, due to the scale and temporal nature of video, the span of video understanding problems and the set of proposed deep learning solutions is arguably wider and more diverse than those of their 2D image siblings. Finding, identifying, and predicting actions are a few of the most salient tasks in this emerging and rapidly evolving field. With a pedagogical emphasis, this tutorial introduces and systematizes fundamental topics, basic concepts, and notable examples in supervised video action understanding. Specifically, we clarify a taxonomy of action problems, catalog and highlight video datasets, describe common video data preparation methods, present the building blocks of state-of-the art deep learning model architectures, and formalize domain-specific metrics to baseline proposed solutions. This tutorial is intended to be accessible to a general computer science audience and assumes a conceptual understanding of supervised learning.