# Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition

arXiv: https://arxiv.org/abs/1702.04710 · HTML: https://mlsanity.com/paper/1702.04710/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2017.

## Built on by (most cited first, 24 of 24)

- [Thin-Plate Spline Motion Model for Image Animation](https://mlsanity.com/paper/2203.14367/) (2022)
- [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022)
- [Vision-Based UAV Self-Positioning in Low-Altitude Urban Environments](https://mlsanity.com/paper/2201.09201/) (2022)
- [Unified Detection of Digital and Physical Face Attacks](https://mlsanity.com/paper/2104.02156/) (2021)
- [LARNet: Lie Algebra Residual Network for Face Recognition](https://mlsanity.com/paper/2103.08147/) (2021)
- [GRATIS: Deep Learning Graph Representation with Task-specific Topology and Multi-dimensional Edge Features](https://mlsanity.com/paper/2211.12482/) (2022)
- [Silhouette based View embeddings for Gait Recognition under Multiple Views](https://mlsanity.com/paper/2108.05524/) (2021)
- [EditID: Training-Free Editable ID Customization for Text-to-Image Generation](https://mlsanity.com/paper/2503.12526/) (2025)
- [PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognition](https://mlsanity.com/paper/2107.11721/) (2021)
- [The Power of Training: How Different Neural Network Setups Influence the Energy Demand](https://mlsanity.com/paper/2401.01851/) (2024)
- [Sample-Level Weighting for Multi-Task Learning with Auxiliary Tasks](https://mlsanity.com/paper/2306.04519/) (2023)
- [Information Maximization for Extreme Pose Face Recognition](https://mlsanity.com/paper/2209.03456/) (2022)
- [Pose Attention-Guided Profile-to-Frontal Face Recognition](https://mlsanity.com/paper/2209.07001/) (2022)
- [Attending Generalizability in Course of Deep Fake Detection by Exploring Multi-task Learning](https://mlsanity.com/paper/2308.13503/) (2023)
- [Attention-guided Progressive Mapping for Profile Face Recognition](https://mlsanity.com/paper/2106.14124/) (2021)
- [Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons](https://mlsanity.com/paper/2311.07955/) (2023)
- [2D-3D Attention and Entropy for Pose Robust 2D Facial Recognition](https://mlsanity.com/paper/2505.09073/) (2025)
- [Profile to Frontal Face Recognition in the Wild Using Coupled Conditional GAN](https://mlsanity.com/paper/2107.13742/) (2021)
- [Multi-Trigger-Key: Towards Multi-Task Privacy Preserving In Deep Learning](https://mlsanity.com/paper/2110.03106/) (2021)
- [Distilling Facial Knowledge With Teacher-Tasks: Semantic-Segmentation-Features For Pose-Invariant Face-Recognition](https://mlsanity.com/paper/2209.01115/) (2022)
- [SASMU: boost the performance of generalized recognition model using synthetic face dataset](https://mlsanity.com/paper/2306.01449/) (2023)
- [Few-shot Personalized Saliency Prediction Based on Interpersonal Gaze Patterns](https://mlsanity.com/paper/2307.02799/) (2023)
- [Towards Large-Scale Pose-Invariant Face Recognition Using Face Defrontalization](https://mlsanity.com/paper/2506.04496/) (2025)
- [Variational Encoder--Multi-Decoder (VE-MD) for Privacy-by-functional-design (Group) Emotion Recognition](https://mlsanity.com/paper/2604.02397/) (2026)

## Abstract (arXiv)

This paper explores multi-task learning (MTL) for face recognition. We answer the questions of how and why MTL can improve the face recognition performance. First, we propose a multi-task Convolutional Neural Network (CNN) for face recognition where identity classification is the main task and pose, illumination, and expression estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss weight to each side task, which is a crucial problem in MTL. Third, we propose a pose-directed multi-task CNN by grouping different poses to learn pose-specific identity features, simultaneously across all poses. Last but not least, we propose an energy-based weight analysis method to explore how CNN-based MTL works. We observe that the side tasks serve as regularizations to disentangle the variations from the learnt identity features. Extensive experiments on the entire Multi-PIE dataset demonstrate the effectiveness of the proposed approach. To the best of our knowledge, this is the first work using all data in Multi-PIE for face recognition. Our approach is also applicable to in-the-wild datasets for pose-invariant face recognition and achieves comparable or better performance than state of the art on LFW, CFP, and IJB-A datasets.