# Supervision by Registration and Triangulation for Landmark Detection

arXiv: https://arxiv.org/abs/2101.09866 · HTML: https://mlsanity.com/paper/2101.09866/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 71 references that are in the map.
- First posted on arXiv 25 Jan 2021.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks](https://mlsanity.com/paper/1711.06753/) (2017)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [Learning Monocular 3D Human Pose Estimation from Multi-view Images](https://mlsanity.com/paper/1803.04775/) (2018)
- [Style Aggregated Network for Facial Landmark Detection](https://mlsanity.com/paper/1803.04108/) (2018)
- [Rethinking on Multi-Stage Networks for Human Pose Estimation](https://mlsanity.com/paper/1901.00148/) (2019)
- [Detect-and-Track: Efficient Pose Estimation in Videos](https://mlsanity.com/paper/1712.09184/) (2017)
- [Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors](https://mlsanity.com/paper/1807.00966/) (2018)
- [Improving Landmark Localization with Semi-Supervised Learning](https://mlsanity.com/paper/1709.01591/) (2017)
- [Disentangling 3D Pose in A Dendritic CNN for Unconstrained 2D Face Alignment](https://mlsanity.com/paper/1802.06713/) (2018)
- [Personalizing Human Video Pose Estimation](https://mlsanity.com/paper/1511.06676/) (2015)
- [Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking](https://mlsanity.com/paper/1907.10815/) (2019)
- [Pose-Invariant Face Alignment with a Single CNN](https://mlsanity.com/paper/1707.06286/) (2017)
- [Joint Multi-view Face Alignment in the Wild](https://mlsanity.com/paper/1708.06023/) (2017)
- [CU-Net: Coupled U-Nets](https://mlsanity.com/paper/1808.06521/) (2018)

## Built on by (most cited first, 30 of 30)

- [Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning](https://mlsanity.com/paper/2203.06541/) (2022)
- [STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection](https://mlsanity.com/paper/2306.02763/) (2023)
- [Sparse to Dense Motion Transfer for Face Image Animation](https://mlsanity.com/paper/2109.00471/) (2021)
- [Sparse to Dense Dynamic 3D Facial Expression Generation](https://mlsanity.com/paper/2105.07463/) (2021)
- [HIH: Towards More Accurate Face Alignment via Heatmap in Heatmap](https://mlsanity.com/paper/2104.03100/) (2021)
- [Graph-based Facial Affect Analysis: A Review](https://mlsanity.com/paper/2103.15599/) (2021)
- [Generating Multiple 4D Expression Transitions by Learning Face Landmark Trajectories](https://mlsanity.com/paper/2208.00050/) (2022)
- [U-LanD: Uncertainty-Driven Video Landmark Detection](https://mlsanity.com/paper/2102.01586/) (2021)
- [LSF-Animation: Label-Free Speech-Driven Facial Animation via Implicit Feature Representation](https://mlsanity.com/paper/2510.21864/) (2025)
- [An Efficient Multitask Neural Network for Face Alignment, Head Pose Estimation and Face Tracking](https://mlsanity.com/paper/2103.07615/) (2021)
- [Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identification](https://mlsanity.com/paper/2312.07338/) (2023)
- [MBW: Multi-view Bootstrapping in the Wild](https://mlsanity.com/paper/2210.01721/) (2022)
- [Mitigating Knowledge Discrepancies among Multiple Datasets for Task-agnostic Unified Face Alignment](https://mlsanity.com/paper/2503.22359/) (2025)
- [Heatmap Regression without Soft-Argmax for Facial Landmark Detection](https://mlsanity.com/paper/2508.14929/) (2025)
- [Generative Landmarks](https://mlsanity.com/paper/2104.04055/) (2021)
- [Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition](https://mlsanity.com/paper/2106.04392/) (2021)
- [Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adapter](https://mlsanity.com/paper/2309.09443/) (2023)
- [GMTR: Graph Matching Transformers](https://mlsanity.com/paper/2311.08141/) (2023)
- [DarkShot: Lighting Dark Images with Low-Compute and High-Quality](https://mlsanity.com/paper/2312.16805/) (2023)
- [TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processing](https://mlsanity.com/paper/2401.05431/) (2024)
- [TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networks](https://mlsanity.com/paper/2401.05432/) (2024)
- [T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Image](https://mlsanity.com/paper/2403.13663/) (2024)
- [eMotion-GAN: A Motion-based GAN for Photorealistic and Facial Expression Preserving Frontal View Synthesis](https://mlsanity.com/paper/2404.09940/) (2024)
- [Deep Learning-Based Quasi-Conformal Surface Registration for Partial 3D Faces Applied to Facial Recognition](https://mlsanity.com/paper/2405.09880/) (2024)
- [Unsupervised learning of Data-driven Facial Expression Coding System (DFECS) using keypoint tracking](https://mlsanity.com/paper/2406.05434/) (2024)
- [A PCA based Keypoint Tracking Approach to Automated Facial Expressions Encoding](https://mlsanity.com/paper/2406.09017/) (2024)
- [Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithms](https://mlsanity.com/paper/2408.17101/) (2024)
- [Towards General Deepfake Detection with Dynamic Curriculum](https://mlsanity.com/paper/2410.11162/) (2024)
- [Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphs](https://mlsanity.com/paper/2410.13409/) (2024)
- [Mask6D: Masked Pose Priors For 6D Object Pose Estimation](https://mlsanity.com/paper/2507.06486/) (2025)

## Abstract (arXiv)

We present Supervision by Registration and Triangulation (SRT), an unsupervised approach that utilizes unlabeled multi-view video to improve the accuracy and precision of landmark detectors. Being able to utilize unlabeled data enables our detectors to learn from massive amounts of unlabeled data freely available and not be limited by the quality and quantity of manual human annotations. To utilize unlabeled data, there are two key observations: (1) the detections of the same landmark in adjacent frames should be coherent with registration, i.e., optical flow. (2) the detections of the same landmark in multiple synchronized and geometrically calibrated views should correspond to a single 3D point, i.e., multi-view consistency. Registration and multi-view consistency are sources of supervision that do not require manual labeling, thus it can be leveraged to augment existing training data during detector training. End-to-end training is made possible by differentiable registration and 3D triangulation modules. Experiments with 11 datasets and a newly proposed metric to measure precision demonstrate accuracy and precision improvements in landmark detection on both images and video. Code is available at https://github.com/D-X-Y/landmark-detection.