# A Unified Deep Framework for Joint 3D Pose Estimation and Action Recognition from a Single RGB Camera

arXiv: https://arxiv.org/abs/1907.06968 · HTML: https://mlsanity.com/paper/1907.06968/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2019.

## Built on by (most cited first, 4 of 4)

- [Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimation](https://mlsanity.com/paper/2108.07181/) (2021)
- [Video-based Human Action Recognition using Deep Learning: A Review](https://mlsanity.com/paper/2208.03775/) (2022)
- [Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Model](https://mlsanity.com/paper/2202.02212/) (2022)
- [Capsules as viewpoint learners for human pose estimation](https://mlsanity.com/paper/2302.06194/) (2023)

## Abstract (arXiv)

We present a deep learning-based multitask framework for joint 3D human pose estimation and action recognition from RGB video sequences. Our approach proceeds along two stages. In the first, we run a real-time 2D pose detector to determine the precise pixel location of important keypoints of the body. A two-stream neural network is then designed and trained to map detected 2D keypoints into 3D poses. In the second, we deploy the Efficient Neural Architecture Search (ENAS) algorithm to find an optimal network architecture that is used for modeling the spatio-temporal evolution of the estimated 3D poses via an image-based intermediate representation and performing action recognition. Experiments on Human3.6M, MSR Action3D and SBU Kinect Interaction datasets verify the effectiveness of the proposed method on the targeted tasks. Moreover, we show that our method requires a low computational budget for training and inference.