# PedRecNet: Multi-task deep neural network for full 3D human pose and orientation estimation

arXiv: https://arxiv.org/abs/2204.11548 · HTML: https://mlsanity.com/paper/2204.11548/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 25 Apr 2022.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay](https://mlsanity.com/paper/1803.09820/) (2018)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Fine-Grained Head Pose Estimation Without Keypoints](https://mlsanity.com/paper/1710.00925/) (2017)
- [VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera](https://mlsanity.com/paper/1705.01583/) (2017)
- [Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach](https://mlsanity.com/paper/1704.02447/) (2017)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning](https://mlsanity.com/paper/1802.09232/) (2018)
- [PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation](https://mlsanity.com/paper/2105.02465/) (2021)
- [3D Human Pose Estimation in the Wild by Adversarial Learning](https://mlsanity.com/paper/1803.09722/) (2018)
- [XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera](https://mlsanity.com/paper/1907.00837/) (2019)
- [Human Pose Regression by Combining Indirect Part Detection and Contextual Information](https://mlsanity.com/paper/1710.02322/) (2017)
- [Deep Kinematic Pose Regression](https://mlsanity.com/paper/1609.05317/) (2016)
- [Multi-task Deep Learning for Real-Time 3D Human Pose Estimation and Action Recognition](https://mlsanity.com/paper/1912.08077/) (2019)
- [Multi-task head pose estimation in-the-wild](https://mlsanity.com/paper/2202.02299/) (2022)
- [Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation](https://mlsanity.com/paper/2107.13994/) (2021)
- [MEBOW: Monocular Estimation of Body Orientation In the Wild](https://mlsanity.com/paper/2011.13688/) (2020)
- [Self-Paced Deep Regression Forests with Consideration on Underrepresented Examples](https://mlsanity.com/paper/2004.01459/) (2020)
- [Designing Deep Convolutional Neural Networks for Continuous Object Orientation Estimation](https://mlsanity.com/paper/1702.01499/) (2017)
- [Nose, eyes and ears: Head pose estimation by locating facial keypoints](https://mlsanity.com/paper/1812.00739/) (2018)
- [Deep Convolutional Neural Network-based Bernoulli Heatmap for Head Pose Estimation](https://mlsanity.com/paper/2005.11780/) (2020)
- [Simple yet efficient real-time pose-based action recognition](https://mlsanity.com/paper/1904.09140/) (2019)

## Built on by (most cited first, 4 of 4)

- [RAG-based Explainable Prediction of Road Users Behaviors for Automated Driving using Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2405.00449/) (2024)
- [Joint Multi-Person Body Detection and Orientation Estimation via One Unified Embedding](https://mlsanity.com/paper/2210.15586/) (2022)
- [Quater-GCN: Enhancing 3D Human Pose Estimation with Orientation and Semi-supervised Training](https://mlsanity.com/paper/2404.19279/) (2024)
- [Experimental Insights Towards Explainable and Interpretable Pedestrian Crossing Prediction](https://mlsanity.com/paper/2312.02872/) (2023)

## Abstract (arXiv)

We present a multitask network that supports various deep neural network based pedestrian detection functions. Besides 2D and 3D human pose, it also supports body and head orientation estimation based on full body bounding box input. This eliminates the need for explicit face recognition. We show that the performance of 3D human pose estimation and orientation estimation is comparable to the state-of-the-art. Since very few data sets exist for 3D human pose and in particular body and head orientation estimation based on full body data, we further show the benefit of particular simulation data to train the network. The network architecture is relatively simple, yet powerful, and easily adaptable for further research and applications.