# Mimetics: Towards Understanding Human Actions Out of Context

arXiv: https://arxiv.org/abs/1912.07249 · HTML: https://mlsanity.com/paper/1912.07249/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2019.

## Built on by (most cited first, 30 of 32)

- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [Evidential Deep Learning for Open Set Action Recognition](https://mlsanity.com/paper/2107.10161/) (2021)
- [Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective](https://mlsanity.com/paper/2110.03095/) (2021)
- [On the Benefits of 3D Pose and Tracking for Human Action Recognition](https://mlsanity.com/paper/2304.01199/) (2023)
- [Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition](https://mlsanity.com/paper/2109.01305/) (2021)
- [HaLP: Hallucinating Latent Positives for Skeleton-based Self-Supervised Learning of Actions](https://mlsanity.com/paper/2304.00387/) (2023)
- [ViA: View-invariant Skeleton Action Representation Learning via Motion Retargeting](https://mlsanity.com/paper/2209.00065/) (2022)
- [Prompt-Guided Zero-Shot Anomaly Action Recognition using Pretrained Deep Skeleton Features](https://mlsanity.com/paper/2303.15167/) (2023)
- [Unified Keypoint-based Action Recognition Framework via Structured Keypoint Pooling](https://mlsanity.com/paper/2303.15270/) (2023)
- [Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances](https://mlsanity.com/paper/2303.10280/) (2023)
- [Spatio-Temporal Action Detection Under Large Motion](https://mlsanity.com/paper/2209.02250/) (2022)
- [SOAR: Scene-debiasing Open-set Action Recognition](https://mlsanity.com/paper/2309.01265/) (2023)
- [NTU-X: An Enhanced Large-scale Dataset for Improving Pose-based Recognition of Subtle Human Actions](https://mlsanity.com/paper/2101.11529/) (2021)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [MimeQA: Towards Socially-Intelligent Nonverbal Foundation Models](https://mlsanity.com/paper/2502.16671/) (2025)
- [GaitContour: Efficient Gait Recognition based on a Contour-Pose Representation](https://mlsanity.com/paper/2311.16497/) (2023)
- [EMAG: Ego-motion Aware and Generalizable 2D Hand Forecasting from Egocentric Videos](https://mlsanity.com/paper/2405.20030/) (2024)
- [Human-Centric Transformer for Domain Adaptive Action Recognition](https://mlsanity.com/paper/2407.10860/) (2024)
- [DEVIAS: Learning Disentangled Video Representations of Action and Scene](https://mlsanity.com/paper/2312.00826/) (2023)
- [Can masking background and object reduce static bias for zero-shot action recognition?](https://mlsanity.com/paper/2501.12681/) (2025)
- [ALBAR: Adversarial Learning approach to mitigate Biases in Action Recognition](https://mlsanity.com/paper/2502.00156/) (2025)
- [Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts](https://mlsanity.com/paper/2303.17595/) (2023)
- [Distillation-guided Representation Learning for Unconstrained Gait Recognition](https://mlsanity.com/paper/2307.14578/) (2023)
- [S3Aug: Segmentation, Sampling, and Shift for Action Recognition](https://mlsanity.com/paper/2310.14556/) (2023)
- [Should I take a walk? Estimating Energy Expenditure from Video Data](https://mlsanity.com/paper/2202.00712/) (2022)
- [Unbiasing through Textual Descriptions: Mitigating Representation Bias in Video Benchmarks](https://mlsanity.com/paper/2503.18637/) (2025)
- [Can Vision Language Models Understand Mimed Actions?](https://mlsanity.com/paper/2506.21586/) (2025)
- [Seeing Beyond the Scene: Analyzing and Mitigating Background Bias in Action Recognition](https://mlsanity.com/paper/2512.17953/) (2025)
- [Separating Shared and Domain-Specific LoRAs for Multi-Domain Learning](https://mlsanity.com/paper/2508.02978/) (2025)
- [Identifying Auxiliary or Adversarial Tasks Using Necessary Condition Analysis for Adversarial Multi-task Video Understanding](https://mlsanity.com/paper/2208.10077/) (2022)

## Abstract (arXiv)

Recent methods for video action recognition have reached outstanding performances on existing benchmarks. However, they tend to leverage context such as scenes or objects instead of focusing on understanding the human action itself. For instance, a tennis field leads to the prediction playing tennis irrespectively of the actions performed in the video. In contrast, humans have a more complete understanding of actions and can recognize them without context. The best example of out-of-context actions are mimes, that people can typically recognize despite missing relevant objects and scenes. In this paper, we propose to benchmark action recognition methods in such absence of context and introduce a novel dataset, Mimetics, consisting of mimed actions for a subset of 50 classes from the Kinetics benchmark. Our experiments show that (a) state-of-the-art 3D convolutional neural networks obtain disappointing results on such videos, highlighting the lack of true understanding of the human actions and (b) models leveraging body language via human pose are less prone to context biases. In particular, we show that applying a shallow neural network with a single temporal convolution over body pose features transferred to the action recognition problem performs surprisingly well compared to 3D action recognition methods.