# Multi-task head pose estimation in-the-wild

arXiv: https://arxiv.org/abs/2202.02299 · HTML: https://mlsanity.com/paper/2202.02299/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 64 references that are in the map.
- First posted on arXiv 4 Feb 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Characterizing and Avoiding Negative Transfer](https://mlsanity.com/paper/1811.09751/) (2018)
- [UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory](https://mlsanity.com/paper/1609.02132/) (2016)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Fine-Grained Head Pose Estimation Without Keypoints](https://mlsanity.com/paper/1710.00925/) (2017)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Transferability and Hardness of Supervised Classification Tasks](https://mlsanity.com/paper/1908.08142/) (2019)
- [Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks](https://mlsanity.com/paper/1711.06753/) (2017)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [Unsupervised Discovery of Object Landmarks as Structural Representations](https://mlsanity.com/paper/1804.04412/) (2018)
- [Facial Landmark Detection: a Literature Survey](https://mlsanity.com/paper/1805.05563/) (2018)
- [An All-In-One Convolutional Neural Network for Face Analysis](https://mlsanity.com/paper/1611.00851/) (2016)
- [Self-supervised learning of a facial attribute embedding from video](https://mlsanity.com/paper/1808.06882/) (2018)
- [Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach](https://mlsanity.com/paper/1706.00906/) (2017)
- [Unsupervised Learning of Landmarks by Descriptor Vector Exchange](https://mlsanity.com/paper/1908.06427/) (2019)
- [Learning Deep Representation for Face Alignment with Auxiliary Attributes](https://mlsanity.com/paper/1408.3967/) (2014)
- [Binarized Convolutional Landmark Localizers for Human Pose Estimation and Face Alignment with Limited Resources](https://mlsanity.com/paper/1703.00862/) (2017)
- [FacePoseNet: Making a Case for Landmark-Free Face Alignment](https://mlsanity.com/paper/1708.07517/) (2017)
- [Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses](https://mlsanity.com/paper/1707.05653/) (2017)
- [Disentangling 3D Pose in A Dendritic CNN for Unconstrained 2D Face Alignment](https://mlsanity.com/paper/1802.06713/) (2018)
- [Robust Facial Landmark Detection under Significant Head Poses and Occlusion](https://mlsanity.com/paper/1709.08127/) (2017)
- [Simultaneous Facial Landmark Detection, Pose and Deformation Estimation under Facial Occlusion](https://mlsanity.com/paper/1709.08130/) (2017)
- [Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation](https://mlsanity.com/paper/1511.07356/) (2015)

## Built on by (most cited first, 23 of 23)

- [3D GAN Inversion with Pose Optimization](https://mlsanity.com/paper/2210.07301/) (2022)
- [SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation](https://mlsanity.com/paper/2308.11509/) (2023)
- [FaceXFormer: A Unified Transformer for Facial Analysis](https://mlsanity.com/paper/2403.12960/) (2024)
- [Shape Preserving Facial Landmarks with Graph Attention Networks](https://mlsanity.com/paper/2210.07233/) (2022)
- [Towards Robust and Unconstrained Full Range of Rotation Head Pose Estimation](https://mlsanity.com/paper/2309.07654/) (2023)
- [Face-MLLM: A Large Face Perception Model](https://mlsanity.com/paper/2410.20717/) (2024)
- [Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data](https://mlsanity.com/paper/2412.01383/) (2024)
- [On the representation and methodology for wide and short range head pose estimation](https://mlsanity.com/paper/2401.05807/) (2024)
- [Towards Unbiased Label Distribution Learning for Facial Pose Estimation Using Anisotropic Spherical Gaussian](https://mlsanity.com/paper/2208.09122/) (2022)
- [DirectMHP: Direct 2D Multi-Person Head Pose Estimation with Full-range Angles](https://mlsanity.com/paper/2302.01110/) (2023)
- [DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignment](https://mlsanity.com/paper/2305.11522/) (2023)
- [FaVChat: Hierarchical Prompt-Query Guided Facial Video Understanding with Data-Efficient GRPO](https://mlsanity.com/paper/2503.09158/) (2025)
- [PedRecNet: Multi-task deep neural network for full 3D human pose and orientation estimation](https://mlsanity.com/paper/2204.11548/) (2022)
- [VGGHeads: 3D Multi Head Alignment with a Large-Scale Synthetic Dataset](https://mlsanity.com/paper/2407.18245/) (2024)
- [Semi-Supervised Unconstrained Head Pose Estimation in the Wild](https://mlsanity.com/paper/2404.02544/) (2024)
- [ORFormer: Occlusion-Robust Transformer for Accurate Facial Landmark Detection](https://mlsanity.com/paper/2412.13174/) (2024)
- [Rank-based No-reference Quality Assessment for Face Swapping](https://mlsanity.com/paper/2406.01884/) (2024)
- [3D Face Alignment Through Fusion of Head Pose Information and Features](https://mlsanity.com/paper/2308.13327/) (2023)
- [On the power of data augmentation for head pose estimation](https://mlsanity.com/paper/2407.05357/) (2024)
- [Bidirectional Regression for Monocular 6DoF Head Pose Estimation and Reference System Alignment](https://mlsanity.com/paper/2407.14136/) (2024)
- [Auditing Demographic Bias in Facial Landmark Detection for Fair Human-Robot Interaction](https://mlsanity.com/paper/2604.06961/) (2026)
- [VGGT-HPE: Reframing Head Pose Estimation as Relative Pose Prediction](https://mlsanity.com/paper/2604.10106/) (2026)
- [DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding](https://mlsanity.com/paper/2610.17015/) (2026)

## Abstract (arXiv)

We present a deep learning-based multi-task approach for head pose estimation in images. We contribute with a network architecture and training strategy that harness the strong dependencies among face pose, alignment and visibility, to produce a top performing model for all three tasks. Our architecture is an encoder-decoder CNN with residual blocks and lateral skip connections. We show that the combination of head pose estimation and landmark-based face alignment significantly improve the performance of the former task. Further, the location of the pose task at the bottleneck layer, at the end of the encoder, and that of tasks depending on spatial information, such as visibility and alignment, in the final decoder layer, also contribute to increase the final performance. In the experiments conducted the proposed model outperforms the state-of-the-art in the face pose and visibility tasks. By including a final landmark regression step it also produces face alignment results on par with the state-of-the-art.