# Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning

arXiv: https://arxiv.org/abs/2203.06541 · HTML: https://mlsanity.com/paper/2203.06541/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 13 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks](https://mlsanity.com/paper/1711.06753/) (2017)
- [TokenPose: Learning Keypoint Tokens for Human Pose Estimation](https://mlsanity.com/paper/2104.03516/) (2021)
- [LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood](https://mlsanity.com/paper/2004.02980/) (2020)
- [Style Aggregated Network for Facial Landmark Detection](https://mlsanity.com/paper/1803.04108/) (2018)
- [Structured Landmark Detection via Topology-Adapting Deep Graph Learning](https://mlsanity.com/paper/2004.08190/) (2020)
- [DeCaFA: Deep Convolutional Cascade for Face Alignment In The Wild](https://mlsanity.com/paper/1904.02549/) (2019)
- [Deep Alignment Network: A convolutional neural network for robust face alignment](https://mlsanity.com/paper/1706.01789/) (2017)
- [ADNet: Leveraging Error-Bias Towards Normal Direction in Face Alignment](https://mlsanity.com/paper/2109.05721/) (2021)
- [Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation](https://mlsanity.com/paper/1908.06440/) (2019)
- [3FabRec: Fast Few-shot Face alignment by Reconstruction](https://mlsanity.com/paper/1911.10448/) (2019)
- [Supervision by Registration and Triangulation for Landmark Detection](https://mlsanity.com/paper/2101.09866/) (2021)
- [Dynamic Attention-controlled Cascaded Shape Regression Exploiting Training Data Augmentation and Fuzzy-set Sample Weighting](https://mlsanity.com/paper/1611.05396/) (2016)
- [Robust Face Alignment by Multi-order High-precision Hourglass Network](https://mlsanity.com/paper/2010.08722/) (2020)

## Built on by (most cited first, 30 of 47)

- [STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection](https://mlsanity.com/paper/2306.02763/) (2023)
- [KeyPoint Relative Position Encoding for Face Recognition](https://mlsanity.com/paper/2403.14852/) (2024)
- [FaceXFormer: A Unified Transformer for Facial Analysis](https://mlsanity.com/paper/2403.12960/) (2024)
- [Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models](https://mlsanity.com/paper/2304.00600/) (2023)
- [Learning Motion-Robust Remote Photoplethysmography through Arbitrary Resolution Videos](https://mlsanity.com/paper/2211.16922/) (2022)
- [Shape Preserving Facial Landmarks with Graph Attention Networks](https://mlsanity.com/paper/2210.07233/) (2022)
- [Self-Supervised Facial Representation Learning with Facial Region Awareness](https://mlsanity.com/paper/2403.02138/) (2024)
- [Mask-FPAN: Semi-Supervised Face Parsing in the Wild With De-Occlusion and UV GAN](https://mlsanity.com/paper/2212.09098/) (2022)
- [Faceptor: A Generalist Model for Face Perception](https://mlsanity.com/paper/2403.09500/) (2024)
- [Text-guided Eyeglasses Manipulation with Spatial Constraints](https://mlsanity.com/paper/2304.12539/) (2023)
- [A Multi-view Mask Contrastive Learning Graph Convolutional Neural Network for Age Estimation](https://mlsanity.com/paper/2407.16234/) (2024)
- [FacialFlowNet: Advancing Facial Optical Flow Estimation with a Diverse Dataset and a Decomposed Model](https://mlsanity.com/paper/2409.05396/) (2024)
- [Generalizable Face Landmarking Guided by Conditional Face Warping](https://mlsanity.com/paper/2404.12322/) (2024)
- [Cascaded Dual Vision Transformer for Accurate Facial Landmark Detection](https://mlsanity.com/paper/2411.07167/) (2024)
- [OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis](https://mlsanity.com/paper/2506.02891/) (2025)
- [Proto-Former: Unified Facial Landmark Detection by Prototype Transformer](https://mlsanity.com/paper/2510.15338/) (2025)
- [KeyPosS: Plug-and-Play Facial Landmark Detection through GPS-Inspired True-Range Multilateration](https://mlsanity.com/paper/2305.16437/) (2023)
- [Rethinking Self-training for Semi-supervised Landmark Detection: A Selection-free Approach](https://mlsanity.com/paper/2404.04556/) (2024)
- [FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields](https://mlsanity.com/paper/2211.11202/) (2022)
- [Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification](https://mlsanity.com/paper/2312.07338/) (2023)
- [Fiducial Focus Augmentation for Facial Landmark Detection](https://mlsanity.com/paper/2402.15044/) (2024)
- [POPoS: Improving Efficient and Robust Facial Landmark Detection with Parallel Optimal Position Search](https://mlsanity.com/paper/2410.09583/) (2024)
- [ORFormer: Occlusion-Robust Transformer for Accurate Facial Landmark Detection](https://mlsanity.com/paper/2412.13174/) (2024)
- [FGTBT: Frequency-Guided Task-Balancing Transformer for Unified Facial Landmark Detection](https://mlsanity.com/paper/2601.12863/) (2026)
- [Mitigating Knowledge Discrepancies among Multiple Datasets for Task-agnostic Unified Face Alignment](https://mlsanity.com/paper/2503.22359/) (2025)
- [Heatmap Regression without Soft-Argmax for Facial Landmark Detection](https://mlsanity.com/paper/2508.14929/) (2025)
- [Event-based Facial Keypoint Alignment via Cross-Modal Fusion Attention and Self-Supervised Multi-Event Representation Learning](https://mlsanity.com/paper/2509.24968/) (2025)
- [Supervision-by-Hallucination-and-Transfer: A Weakly-Supervised Approach for Robust and Precise Facial Landmark Detection](https://mlsanity.com/paper/2601.12919/) (2026)
- [OccFace: Unified Occlusion-Aware Facial Landmark Detection with Per-Point Visibility](https://mlsanity.com/paper/2602.10728/) (2026)
- [FreqFLD: Towards All-in-One Facial Landmark Detection via Frequency Modulation](https://mlsanity.com/paper/2609.10278/) (2026)

## Abstract (arXiv)

Heatmap regression methods have dominated face alignment area in recent years while they ignore the inherent relation between different landmarks. In this paper, we propose a Sparse Local Patch Transformer (SLPT) for learning the inherent relation. The SLPT generates the representation of each single landmark from a local patch and aggregates them by an adaptive inherent relation based on the attention mechanism. The subpixel coordinate of each landmark is predicted independently based on the aggregated feature. Moreover, a coarse-to-fine framework is further introduced to incorporate with the SLPT, which enables the initial landmarks to gradually converge to the target facial landmarks using fine-grained features from dynamically resized local patches. Extensive experiments carried out on three popular benchmarks, including WFLW, 300W and COFW, demonstrate that the proposed method works at the state-of-the-art level with much less computational complexity by learning the inherent relation between facial landmarks. The code is available at the project website.