# SwinFace: A Multi-task Transformer for Face Recognition, Expression Recognition, Age Estimation and Attribute Estimation

arXiv: https://arxiv.org/abs/2308.11509 · HTML: https://mlsanity.com/paper/2308.11509/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 80 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/1708.08197/) (2017)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition](https://mlsanity.com/paper/1905.04075/) (2019)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [TransFER: Learning Relation-aware Facial Expression Representations with Transformers](https://mlsanity.com/paper/2108.11116/) (2021)
- [Face Transformer for Recognition](https://mlsanity.com/paper/2103.14803/) (2021)
- [An All-In-One Convolutional Neural Network for Face Analysis](https://mlsanity.com/paper/1611.00851/) (2016)
- [Moving Window Regression: A Novel Approach to Ordinal Regression](https://mlsanity.com/paper/2203.13122/) (2022)
- [BridgeNet: A Continuity-Aware Probabilistic Network for Age Estimation](https://mlsanity.com/paper/1904.03358/) (2019)
- [Joint Hand-object 3D Reconstruction from a Single Image with Cross-branch Feature Fusion](https://mlsanity.com/paper/2006.15561/) (2020)
- [Covariance Pooling For Facial Expression Recognition](https://mlsanity.com/paper/1805.04855/) (2018)

## Built on by (most cited first, 30 of 39)

- [FaceXFormer: A Unified Transformer for Facial Analysis](https://mlsanity.com/paper/2403.12960/) (2024)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [Faceptor: A Generalist Model for Face Perception](https://mlsanity.com/paper/2403.09500/) (2024)
- [Face-MLLM: A Large Face Perception Model](https://mlsanity.com/paper/2410.20717/) (2024)
- [Open-Set Biometrics: Beyond Good Closed-Set Models](https://mlsanity.com/paper/2407.16133/) (2024)
- [Face-LLaVA: Facial Expression and Attribute Understanding through Instruction Tuning](https://mlsanity.com/paper/2504.07198/) (2025)
- [ViT-FIQA: Assessing Face Image Quality using Vision Transformers](https://mlsanity.com/paper/2508.13957/) (2025)
- [Beyond Real Faces: Synthetic Datasets Can Achieve Reliable Recognition Performance without Privacy Compromise](https://mlsanity.com/paper/2510.17372/) (2025)
- [FaVChat: Hierarchical Prompt-Query Guided Facial Video Understanding with Data-Efficient GRPO](https://mlsanity.com/paper/2503.09158/) (2025)
- [Task-adaptive Q-Face](https://mlsanity.com/paper/2405.09059/) (2024)
- [Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey](https://mlsanity.com/paper/2412.17616/) (2024)
- [OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis](https://mlsanity.com/paper/2506.02891/) (2025)
- [FROQ: Observing Face Recognition Models for Efficient Quality Assessment](https://mlsanity.com/paper/2509.17689/) (2025)
- [GroupFace: Imbalanced Age Estimation Based on Multi-hop Attention Graph Convolutional Network and Group-aware Margin Optimization](https://mlsanity.com/paper/2412.11450/) (2024)
- [FaceInsight: A Multimodal Large Language Model for Face Perception](https://mlsanity.com/paper/2504.15624/) (2025)
- [FLUXSynID: A Framework for Identity-Controlled Synthetic Face Generation with Document and Live Images](https://mlsanity.com/paper/2505.07530/) (2025)
- [AI-KD: Towards Alignment Invariant Face Image Quality Assessment Using Knowledge Distillation](https://mlsanity.com/paper/2404.09555/) (2024)
- [HPE-CogVLM: Advancing Vision Language Models with a Head Pose Grounding Task](https://mlsanity.com/paper/2406.01914/) (2024)
- [LRA-GNN: Latent Relation-Aware Graph Neural Network with Initial and Dynamic Residual for Facial Age Estimation](https://mlsanity.com/paper/2502.05423/) (2025)
- [Overcoming Occlusions in the Wild: A Multi-Task Age Head Approach to Age Estimation](https://mlsanity.com/paper/2506.13445/) (2025)
- [Reverse Personalization](https://mlsanity.com/paper/2512.22984/) (2025)
- [Multi-Track Multimodal Learning on iMiGUE: Micro-Gesture and Emotion Recognition](https://mlsanity.com/paper/2512.23291/) (2025)
- [Facial Attractiveness Prediction in Live Streaming: A New Benchmark and Multi-modal Method](https://mlsanity.com/paper/2501.02509/) (2025)
- [Unified Face Matching and Physical-Digital Spoofing Attack Detection](https://mlsanity.com/paper/2501.09635/) (2025)
- [Archival Faces: Detection of Faces in Digitized Historical Documents](https://mlsanity.com/paper/2504.00558/) (2025)
- [SFANet: Spatial-Frequency Attention Network for Deepfake Detection](https://mlsanity.com/paper/2510.04630/) (2025)
- [Integrating ConvNeXt and Vision Transformers for Enhancing Facial Age Estimation](https://mlsanity.com/paper/2511.00123/) (2025)
- [FLUID: Training-Free Face De-identification via Latent Identity Substitution](https://mlsanity.com/paper/2511.17005/) (2025)
- [The Good, the Better, and the Best: Improving the Discriminability of Face Embeddings through Attribute-aware Learning](https://mlsanity.com/paper/2603.15062/) (2026)
- [Adversarial Camouflage](https://mlsanity.com/paper/2603.21867/) (2026)

## Abstract (arXiv)

In recent years, vision transformers have been introduced into face recognition and analysis and have achieved performance breakthroughs. However, most previous methods generally train a single model or an ensemble of models to perform the desired task, which ignores the synergy among different tasks and fails to achieve improved prediction accuracy, increased data efficiency, and reduced training time. This paper presents a multi-purpose algorithm for simultaneous face recognition, facial expression recognition, age estimation, and face attribute estimation (40 attributes including gender) based on a single Swin Transformer. Our design, the SwinFace, consists of a single shared backbone together with a subnet for each set of related tasks. To address the conflicts among multiple tasks and meet the different demands of tasks, a Multi-Level Channel Attention (MLCA) module is integrated into each task-specific analysis subnet, which can adaptively select the features from optimal levels and channels to perform the desired tasks. Extensive experiments show that the proposed model has a better understanding of the face and achieves excellent performance for all tasks. Especially, it achieves 90.97% accuracy on RAF-DB and 0.22 ε-error on CLAP2015, which are state-of-the-art results on facial expression recognition and age estimation respectively. The code and models will be made publicly available at https://github.com/lxq1000/SwinFace.