# Rendering of Eyes for Eye-Shape Registration and Gaze Estimation

arXiv: https://arxiv.org/abs/1505.05916 · HTML: https://mlsanity.com/paper/1505.05916/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2015.

## Built on by (most cited first, 30 of 49)

- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark](https://mlsanity.com/paper/2104.12668/) (2021)
- [Automatic Gaze Analysis: A Survey of Deep Learning based Approaches](https://mlsanity.com/paper/2108.05479/) (2021)
- [RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis](https://mlsanity.com/paper/2205.07058/) (2022)
- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [SCAMPS: Synthetics for Camera Measurement of Physiological Signals](https://mlsanity.com/paper/2206.04197/) (2022)
- [NViSII: A Scriptable Tool for Photorealistic Image Generation](https://mlsanity.com/paper/2105.13962/) (2021)
- [M-SENA: An Integrated Platform for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2203.12441/) (2022)
- [LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis](https://mlsanity.com/paper/2308.10713/) (2023)
- [Do I Have Your Attention: A Large Scale Engagement Prediction Dataset and Baselines](https://mlsanity.com/paper/2302.00431/) (2023)
- [TinyTracker: Ultra-Fast and Ultra-Low-Power Edge Vision In-Sensor for Gaze Estimation](https://mlsanity.com/paper/2307.07813/) (2023)
- [An LLM Benchmark for Addressee Recognition in Multi-modal Multi-party Dialogue](https://mlsanity.com/paper/2501.16643/) (2025)
- [EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes](https://mlsanity.com/paper/2206.08428/) (2022)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [Learning-by-Novel-View-Synthesis for Full-Face Appearance-Based 3D Gaze Estimation](https://mlsanity.com/paper/2201.07927/) (2022)
- [An End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices](https://mlsanity.com/paper/2307.00122/) (2023)
- [Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges](https://mlsanity.com/paper/2305.14080/) (2023)
- [Motion Matters: Neural Motion Transfer for Better Camera Physiological Measurement](https://mlsanity.com/paper/2303.12059/) (2023)
- [EFE: End-to-end Frame-to-Gaze Estimation](https://mlsanity.com/paper/2305.05526/) (2023)
- [Where Do Deep Fakes Look? Synthetic Face Detection via Gaze Tracking](https://mlsanity.com/paper/2101.01165/) (2021)
- [MoodCapture: Depression Detection Using In-the-Wild Smartphone Images](https://mlsanity.com/paper/2402.16182/) (2024)
- [UniGaze: Towards Universal Gaze Estimation via Large-scale Pre-Training](https://mlsanity.com/paper/2502.02307/) (2025)
- [Engagement Measurement Based on Facial Landmarks and Spatial-Temporal Graph Convolutional Networks](https://mlsanity.com/paper/2403.17175/) (2024)
- [Few-shot Geometry-Aware Keypoint Localization](https://mlsanity.com/paper/2303.17216/) (2023)
- [High-Fidelity Eye Animatable Neural Radiance Fields for Human Face](https://mlsanity.com/paper/2308.00773/) (2023)
- [CUDA-GHR: Controllable Unsupervised Domain Adaptation for Gaze and Head Redirection](https://mlsanity.com/paper/2106.10852/) (2021)
- [FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning](https://mlsanity.com/paper/2209.06692/) (2022)
- [Mesh-Tension Driven Expression-Based Wrinkles for Synthetic Faces](https://mlsanity.com/paper/2210.03529/) (2022)
- [Gaze-enhanced Crossmodal Embeddings for Emotion Recognition](https://mlsanity.com/paper/2205.00129/) (2022)
- [Precise localization of corneal reflections in eye images using deep learning trained on synthetic data](https://mlsanity.com/paper/2304.05673/) (2023)

## Abstract (arXiv)

Images of the eye are key in several computer vision problems, such as shape registration and gaze estimation. Recent large-scale supervised methods for these problems require time-consuming data collection and manual annotation, which can be unreliable. We propose synthesizing perfectly labelled photo-realistic training data in a fraction of the time. We used computer graphics techniques to build a collection of dynamic eye-region models from head scan geometry. These were randomly posed to synthesize close-up eye images for a wide range of head poses, gaze directions, and illumination conditions. We used our model's controllability to verify the importance of realistic illumination and shape variations in eye-region training data. Finally, we demonstrate the benefits of our synthesized training data (SynthesEyes) by out-performing state-of-the-art methods for eye-shape registration as well as cross-dataset appearance-based gaze estimation in the wild.