# GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2411.12981 · HTML: https://mlsanity.com/paper/2411.12981/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 64 references that are in the map.
- First posted on arXiv 20 Nov 2024.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians](https://mlsanity.com/paper/2312.02069/) (2023)
- [Capture, Learning, and Synthesis of 3D Speaking Styles](https://mlsanity.com/paper/1905.03079/) (2019)
- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1804.03786/) (2018)
- [Eye Tracking for Everyone](https://mlsanity.com/paper/1606.05814/) (2016)
- [Appearance-Based Gaze Estimation in the Wild](https://mlsanity.com/paper/1504.02863/) (2015)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation](https://mlsanity.com/paper/2007.15837/) (2020)
- [It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1611.08860/) (2016)
- [FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding](https://mlsanity.com/paper/2312.02214/) (2023)
- [textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Driving](https://mlsanity.com/paper/2405.20323/) (2024)
- [Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark](https://mlsanity.com/paper/2104.12668/) (2021)
- [Few-Shot Adaptive Gaze Estimation](https://mlsanity.com/paper/1905.01941/) (2019)
- [Gaze Estimation using Transformer](https://mlsanity.com/paper/2105.14424/) (2021)
- [SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces](https://mlsanity.com/paper/2306.10799/) (2023)
- [HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting](https://mlsanity.com/paper/2312.02902/) (2023)
- [FaceX-Zoo: A PyTorch Toolbox for Face Recognition](https://mlsanity.com/paper/2101.04407/) (2021)
- [GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians](https://mlsanity.com/paper/2402.10483/) (2024)

## Built on by (most cited first, 9 of 9)

- [ParkGaussian: Surround-view 3D Gaussian Splatting for Autonomous Parking](https://mlsanity.com/paper/2601.01386/) (2026)
- [DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillation](https://mlsanity.com/paper/2503.18159/) (2025)
- [OmniIndoor3D: Comprehensive Indoor 3D Reconstruction](https://mlsanity.com/paper/2505.20610/) (2025)
- [DNRSelect: Active Best View Selection for Deferred Neural Rendering](https://mlsanity.com/paper/2501.12150/) (2025)
- [RSATalker: Realistic Socially-Aware Talking Head Generation for Multi-Turn Conversation](https://mlsanity.com/paper/2601.10606/) (2026)
- [GazeShift: Unsupervised Gaze Estimation and Dataset for VR](https://mlsanity.com/paper/2603.07832/) (2026)
- [GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors](https://mlsanity.com/paper/2606.08440/) (2026)
- [GazePrior: Zero-Shot AR/VR Eye Tracking via Learned 3D Gaze Reconstruction](https://mlsanity.com/paper/2605.22359/) (2026)
- [GazeDiT: Gaze-Accurate Diffusion Image Generation for Eye Tracking via Spatial Conditioning](https://mlsanity.com/paper/2609.17814/) (2026)

## Abstract (arXiv)

Gaze estimation encounters generalization challenges when dealing with out-of-distribution data. To address this problem, recent methods use neural radiance fields (NeRF) to generate augmented data. However, existing methods based on NeRF are computationally expensive and lack facial details. 3D Gaussian Splatting (3DGS) has become the prevailing representation of neural fields. While 3DGS has been extensively examined in head avatars, it faces challenges with accurate gaze control and generalization across different subjects. In this work, we propose GazeGaussian, the first high-fidelity gaze redirection method that uses a two-stream 3DGS model to represent the face and eye regions separately. Leveraging the unstructured nature of 3DGS, we develop a novel representation of the eye for rigid eye rotation based on the target gaze direction. To enable synthesis generalization across various subjects, we integrate an expression-guided module to inject subject-specific information into the neural renderer. Comprehensive experiments show that GazeGaussian outperforms existing methods in rendering speed, gaze redirection accuracy, and facial synthesis across multiple datasets. The code is available at: https://ucwxb.github.io/GazeGaussian.