# Self-Learning Transformations for Improving Gaze and Head Redirection

arXiv: https://arxiv.org/abs/2010.12307 · HTML: https://mlsanity.com/paper/2010.12307/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 28 of 28)

- [Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark](https://mlsanity.com/paper/2104.12668/) (2021)
- [PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation](https://mlsanity.com/paper/2103.13173/) (2021)
- [Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptation](https://mlsanity.com/paper/2107.13780/) (2021)
- [Automatic Gaze Analysis: A Survey of Deep Learning based Approaches](https://mlsanity.com/paper/2108.05479/) (2021)
- [EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes](https://mlsanity.com/paper/2206.08428/) (2022)
- [Contrastive Representation Learning for Gaze Estimation](https://mlsanity.com/paper/2210.13404/) (2022)
- [Learning-by-Novel-View-Synthesis for Full-Face Appearance-Based 3D Gaze Estimation](https://mlsanity.com/paper/2201.07927/) (2022)
- [GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields](https://mlsanity.com/paper/2212.04823/) (2022)
- [MTGLS: Multi-Task Gaze Estimation with Limited Supervision](https://mlsanity.com/paper/2110.12100/) (2021)
- [Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control](https://mlsanity.com/paper/2208.02210/) (2022)
- [NeRF-Gaze: A Head-Eye Redirection Parametric Model for Gaze Estimation](https://mlsanity.com/paper/2212.14710/) (2022)
- [Rotation-Constrained Cross-View Feature Fusion for Multi-View Appearance-based Gaze Estimation](https://mlsanity.com/paper/2305.12704/) (2023)
- [GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting](https://mlsanity.com/paper/2411.12981/) (2024)
- [LatentGaze: Cross-Domain Gaze Estimation through Gaze-Aware Analytic Latent Code Manipulation](https://mlsanity.com/paper/2209.10171/) (2022)
- [UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training](https://mlsanity.com/paper/2502.02307/) (2025)
- [ReDirTrans: Latent-to-Latent Translation for Gaze and Head Redirection](https://mlsanity.com/paper/2305.11452/) (2023)
- [High-Fidelity Eye Animatable Neural Radiance Fields for Human Face](https://mlsanity.com/paper/2308.00773/) (2023)
- [CUDA-GHR: Controllable Unsupervised Domain Adaptation for Gaze and Head Redirection](https://mlsanity.com/paper/2106.10852/) (2021)
- [PrivateGaze: Preserving User Privacy in Black-box Mobile Gaze Tracking Services](https://mlsanity.com/paper/2408.00950/) (2024)
- [Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement](https://mlsanity.com/paper/2305.16140/) (2023)
- [TextGaze: Gaze-Controllable Face Generation with Natural Language](https://mlsanity.com/paper/2404.17486/) (2024)
- [Learning Video-independent Eye Contact Segmentation from In-the-Wild Videos](https://mlsanity.com/paper/2210.02033/) (2022)
- [AUEditNet: Dual-Branch Facial Action Unit Intensity Manipulation with Implicit Disentanglement](https://mlsanity.com/paper/2404.05063/) (2024)
- [Angle Range and Identity Similarity Enhanced Gaze and Head Redirection based on Synthetic data](https://mlsanity.com/paper/2309.05214/) (2023)
- [Roll Your Eyes: Gaze Redirection via Explicit 3D Eyeball Rotation](https://mlsanity.com/paper/2508.06136/) (2025)
- [GazePrior: Zero-Shot AR/VR Eye Tracking via Learned 3D Gaze Reconstruction](https://mlsanity.com/paper/2605.22359/) (2026)
- [Data-driven Head Motion Generation through Natural Gaze-Head Coordination](https://mlsanity.com/paper/2605.25810/) (2026)
- [GazeDiT: Gaze-Accurate Diffusion Image Generation for Eye Tracking via Spatial Conditioning](https://mlsanity.com/paper/2609.17814/) (2026)

## Abstract (arXiv)

Many computer vision tasks rely on labeled data. Rapid progress in generative modeling has led to the ability to synthesize photorealistic images. However, controlling specific aspects of the generation process such that the data can be used for supervision of downstream tasks remains challenging. In this paper we propose a novel generative model for images of faces, that is capable of producing high-quality images under fine-grained control over eye gaze and head orientation angles. This requires the disentangling of many appearance related factors including gaze and head orientation but also lighting, hue etc. We propose a novel architecture which learns to discover, disentangle and encode these extraneous variations in a self-learned manner. We further show that explicitly disentangling task-irrelevant factors results in more accurate modelling of gaze and head orientation. A novel evaluation scheme shows that our method improves upon the state-of-the-art in redirection accuracy and disentanglement between gaze direction and head orientation changes. Furthermore, we show that in the presence of limited amounts of real-world training data, our method allows for improvements in the downstream task of semi-supervised cross-dataset gaze estimation. Please check our project page at: https://ait.ethz.ch/projects/2020/STED-gaze/