# A Recurrent Encoder-Decoder Network for Sequential Face Alignment

arXiv: https://arxiv.org/abs/1608.05477 · HTML: https://mlsanity.com/paper/1608.05477/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2016.

## Built on by (most cited first, 9 of 9)

- [Learning Enriched Features for Fast Image Restoration and Enhancement](https://mlsanity.com/paper/2205.01649/) (2022)
- [Supervision by Registration and Triangulation for Landmark Detection](https://mlsanity.com/paper/2101.09866/) (2021)
- [SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction](https://mlsanity.com/paper/2106.03021/) (2021)
- [3D Pose Estimation and Future Motion Prediction from 2D Images](https://mlsanity.com/paper/2111.13285/) (2021)
- [Reasoning Structural Relation for Occlusion-Robust Facial Landmark Localization](https://mlsanity.com/paper/2112.10087/) (2021)
- [OVeNet: Offset Vector Network for Semantic Segmentation](https://mlsanity.com/paper/2303.14516/) (2023)
- [1DFormer: a Transformer Architecture Learning 1D Landmark Representations for Facial Landmark Tracking](https://mlsanity.com/paper/2311.00241/) (2023)
- [Unsupervised learning of Data-driven Facial Expression Coding System (DFECS) using keypoint tracking](https://mlsanity.com/paper/2406.05434/) (2024)
- [Semantic Segmentation for Real-World and Synthetic Vehicle's Forward-Facing Camera Images](https://mlsanity.com/paper/2407.05452/) (2024)

## Abstract (arXiv)

We propose a novel recurrent encoder-decoder network model for real-time video-based face alignment. Our proposed model predicts 2D facial point maps regularized by a regression loss, while uniquely exploiting recurrent learning at both spatial and temporal dimensions. At the spatial level, we add a feedback loop connection between the combined output response map and the input, in order to enable iterative coarse-to-fine face alignment using a single network model. At the temporal level, we first decouple the features in the bottleneck of the network into temporal-variant factors, such as pose and expression, and temporal-invariant factors, such as identity information. Temporal recurrent learning is then applied to the decoupled temporal-variant features, yielding better generalization and significantly more accurate results at test time. We perform a comprehensive experimental analysis, showing the importance of each component of our proposed model, as well as superior results over the state-of-the-art in standard datasets.