# Synthetic Humans for Action Recognition from Unseen Viewpoints

arXiv: https://arxiv.org/abs/1912.04070 · HTML: https://mlsanity.com/paper/1912.04070/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2019.

## Built on by (most cited first, 30 of 46)

- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis](https://mlsanity.com/paper/2109.05488/) (2021)
- [SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation](https://mlsanity.com/paper/2304.10417/) (2023)
- [Implicit Neural Representations for Variable Length Human Motion Generation](https://mlsanity.com/paper/2203.13694/) (2022)
- [Body Meshes as Points](https://mlsanity.com/paper/2105.02467/) (2021)
- [Going Beyond Nouns With Vision & Language Models Using Synthetic Data](https://mlsanity.com/paper/2303.17590/) (2023)
- [On the Benefits of 3D Pose and Tracking for Human Action Recognition](https://mlsanity.com/paper/2304.01199/) (2023)
- [Synthetic Data in Human Analysis: A Survey](https://mlsanity.com/paper/2208.09191/) (2022)
- [Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Data](https://mlsanity.com/paper/2112.00054/) (2021)
- [ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints](https://mlsanity.com/paper/2112.03905/) (2021)
- [Just Add π! Pose Induced Video Transformers for Understanding Activities of Daily Living](https://mlsanity.com/paper/2311.18840/) (2023)
- [InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation](https://mlsanity.com/paper/2101.03049/) (2021)
- [Let's Play for Action: Recognizing Activities of Daily Living by Learning from Life Simulation Video Games](https://mlsanity.com/paper/2107.05617/) (2021)
- [PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Data](https://mlsanity.com/paper/2212.04821/) (2022)
- [Implicit field supervision for robust non-rigid shape matching](https://mlsanity.com/paper/2203.07694/) (2022)
- [T2LM: Long-Term 3D Human Motion Generation from Multiple Sentences](https://mlsanity.com/paper/2406.00636/) (2024)
- [Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data](https://mlsanity.com/paper/2503.01167/) (2025)
- [Multi-Person 3D Pose and Shape Estimation via Inverse Kinematics and Refinement](https://mlsanity.com/paper/2210.13529/) (2022)
- [Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment](https://mlsanity.com/paper/2402.04599/) (2024)
- [M3Act: Learning from Synthetic Human Group Activities](https://mlsanity.com/paper/2306.16772/) (2023)
- [Learning Human Action Recognition Representations Without Real Humans](https://mlsanity.com/paper/2311.06231/) (2023)
- [A Benchmark of Video-Based Clothes-Changing Person Re-Identification](https://mlsanity.com/paper/2211.11165/) (2022)
- [Learning from Synthetic Data for Visual Grounding](https://mlsanity.com/paper/2403.13804/) (2024)
- [SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation](https://mlsanity.com/paper/2504.05925/) (2025)
- [Unsupervised View-Invariant Human Posture Representation](https://mlsanity.com/paper/2109.08730/) (2021)
- [Music-driven Dance Regeneration with Controllable Key Pose Constraints](https://mlsanity.com/paper/2207.03682/) (2022)
- [Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy](https://mlsanity.com/paper/2410.17098/) (2024)
- [DynaDog+T: A Parametric Animal Model for Synthetic Canine Image Generation](https://mlsanity.com/paper/2107.07330/) (2021)
- [Multimodal Generation of Novel Action Appearances for Synthetic-to-Real Recognition of Activities of Daily Living](https://mlsanity.com/paper/2208.01910/) (2022)

## Abstract (arXiv)

Although synthetic training data has been shown to be beneficial for tasks such as human pose estimation, its use for RGB human action recognition is relatively unexplored. Our goal in this work is to answer the question whether synthetic humans can improve the performance of human action recognition, with a particular focus on generalization to unseen viewpoints. We make use of the recent advances in monocular 3D human body reconstruction from real action sequences to automatically render synthetic training videos for the action labels. We make the following contributions: (i) we investigate the extent of variations and augmentations that are beneficial to improving performance at new viewpoints. We consider changes in body shape and clothing for individuals, as well as more action relevant augmentations such as non-uniform frame sampling, and interpolating between the motion of individuals performing the same action; (ii) We introduce a new data generation methodology, SURREACT, that allows training of spatio-temporal CNNs for action classification; (iii) We substantially improve the state-of-the-art action recognition performance on the NTU RGB+D and UESTC standard human action multi-view benchmarks; Finally, (iv) we extend the augmentation approach to in-the-wild videos from a subset of the Kinetics dataset to investigate the case when only one-shot training data is available, and demonstrate improvements in this case as well.