# Towards Accurate Facial Landmark Detection via Cascaded Transformers

arXiv: https://arxiv.org/abs/2208.10808 · HTML: https://mlsanity.com/paper/2208.10808/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 23 Aug 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [TransPose: Keypoint Localization via Transformer](https://mlsanity.com/paper/2012.14214/) (2020)
- [Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks](https://mlsanity.com/paper/1711.06753/) (2017)
- [Pose Recognition with Cascade Transformers](https://mlsanity.com/paper/2104.06976/) (2021)
- [LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood](https://mlsanity.com/paper/2004.02980/) (2020)
- [Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild](https://mlsanity.com/paper/2003.03771/) (2020)
- [Facial Landmark Detection: a Literature Survey](https://mlsanity.com/paper/1805.05563/) (2018)
- [Style Aggregated Network for Facial Landmark Detection](https://mlsanity.com/paper/1803.04108/) (2018)
- [Structured Landmark Detection via Topology-Adapting Deep Graph Learning](https://mlsanity.com/paper/2004.08190/) (2020)
- [TFPose: Direct Human Pose Estimation with Transformers](https://mlsanity.com/paper/2103.15320/) (2021)
- [Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection](https://mlsanity.com/paper/1908.02116/) (2019)
- [Quantized Densely Connected U-Nets for Efficient Landmark Localization](https://mlsanity.com/paper/1808.02194/) (2018)
- [Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark Detection](https://mlsanity.com/paper/1903.10661/) (2019)
- [When Liebig's Barrel Meets Facial Landmark Detection: A Practical Model](https://mlsanity.com/paper/2105.13150/) (2021)

## Built on by (most cited first, 30 of 31)

- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection](https://mlsanity.com/paper/2306.02763/) (2023)
- [DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices](https://mlsanity.com/paper/2303.13325/) (2023)
- [Shape Preserving Facial Landmarks with Graph Attention Networks](https://mlsanity.com/paper/2210.07233/) (2022)
- [Self-Supervised Facial Representation Learning with Facial Region Awareness](https://mlsanity.com/paper/2403.02138/) (2024)
- [CRT-6D: Fast 6D Object Pose Estimation with Cascaded Refinement Transformers](https://mlsanity.com/paper/2210.11718/) (2022)
- [Faceptor: A Generalist Model for Face Perception](https://mlsanity.com/paper/2403.09500/) (2024)
- [Generalizable Face Landmarking Guided by Conditional Face Warping](https://mlsanity.com/paper/2404.12322/) (2024)
- [FLAIR: A Conditional Diffusion Framework with Applications to Face Video Restoration](https://mlsanity.com/paper/2311.15445/) (2023)
- [Cascaded Dual Vision Transformer for Accurate Facial Landmark Detection](https://mlsanity.com/paper/2411.07167/) (2024)
- [LaTo: Landmark-tokenized Diffusion Transformer for Fine-grained Human Face Editing](https://mlsanity.com/paper/2509.25731/) (2025)
- [Proto-Former: Unified Facial Landmark Detection by Prototype Transformer](https://mlsanity.com/paper/2510.15338/) (2025)
- [KeyPosS: Plug-and-Play Facial Landmark Detection through GPS-Inspired True-Range Multilateration](https://mlsanity.com/paper/2305.16437/) (2023)
- [Automated Detection of Cat Facial Landmarks](https://mlsanity.com/paper/2310.09793/) (2023)
- [Rethinking Self-training for Semi-supervised Landmark Detection: A Selection-free Approach](https://mlsanity.com/paper/2404.04556/) (2024)
- [Fiducial Focus Augmentation for Facial Landmark Detection](https://mlsanity.com/paper/2402.15044/) (2024)
- [ORFormer: Occlusion-Robust Transformer for Accurate Facial Landmark Detection](https://mlsanity.com/paper/2412.13174/) (2024)
- [FGTBT: Frequency-Guided Task-Balancing Transformer for Unified Facial Landmark Detection](https://mlsanity.com/paper/2601.12863/) (2026)
- [Heatmap Regression without Soft-Argmax for Facial Landmark Detection](https://mlsanity.com/paper/2508.14929/) (2025)
- [Supervision-by-Hallucination-and-Transfer: A Weakly-Supervised Approach for Robust and Precise Facial Landmark Detection](https://mlsanity.com/paper/2601.12919/) (2026)
- [SimFLE: Simple Facial Landmark Encoding for Self-Supervised Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2303.07648/) (2023)
- [LocalEyenet: Deep Attention framework for Localization of Eyes](https://mlsanity.com/paper/2303.12728/) (2023)
- [Coordinated Transformer with Position & Sample-aware Central Loss for Anatomical Landmark Detection](https://mlsanity.com/paper/2305.11338/) (2023)
- [Deformation Robust Text Spotting with Geometric Prior](https://mlsanity.com/paper/2308.16404/) (2023)
- [1DFormer: a Transformer Architecture Learning 1D Landmark Representations for Facial Landmark Tracking](https://mlsanity.com/paper/2311.00241/) (2023)
- [Adaptive Query Prompting for Multi-Domain Landmark Detection](https://mlsanity.com/paper/2404.01194/) (2024)
- [Densemarks: Learning Canonical Embeddings for Human Heads Images via Point Tracks](https://mlsanity.com/paper/2511.02830/) (2025)
- [A2BFR: Attribute-Aware Blind Face Restoration](https://mlsanity.com/paper/2603.29423/) (2026)
- [Towards Unified Dynamic Face Landmark Detection](https://mlsanity.com/paper/2608.10346/) (2026)
- [FAHCD-Net: Frequency-Adaptive Heatmap-Conditional Diffusion Networks for Robust Facial Landmark Detection](https://mlsanity.com/paper/2609.16842/) (2026)

## Abstract (arXiv)

Accurate facial landmarks are essential prerequisites for many tasks related to human faces. In this paper, an accurate facial landmark detector is proposed based on cascaded transformers. We formulate facial landmark detection as a coordinate regression task such that the model can be trained end-to-end. With self-attention in transformers, our model can inherently exploit the structured relationships between landmarks, which would benefit landmark detection under challenging conditions such as large pose and occlusion. During cascaded refinement, our model is able to extract the most relevant image features around the target landmark for coordinate prediction, based on deformable attention mechanism, thus bringing more accurate alignment. In addition, we propose a novel decoder that refines image features and landmark positions simultaneously. With few parameter increasing, the detection performance improves further. Our model achieves new state-of-the-art performance on several standard facial landmark detection benchmarks, and shows good generalization ability in cross-dataset evaluation.