# Introducing Explicit Gaze Constraints to Face Swapping

arXiv: https://arxiv.org/abs/2305.16138 · HTML: https://mlsanity.com/paper/2305.16138/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 28 of 28)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [SimSwap: An Efficient Framework For High Fidelity Face Swapping](https://mlsanity.com/paper/2106.06340/) (2021)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [DeepFaceLab: Integrated, flexible and extensible face-swapping framework](https://mlsanity.com/paper/2005.05535/) (2020)
- [Fast Face-swap Using Convolutional Neural Networks](https://mlsanity.com/paper/1611.09577/) (2016)
- [One Shot Face Swapping on Megapixels](https://mlsanity.com/paper/2105.04932/) (2021)
- [HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping](https://mlsanity.com/paper/2106.09965/) (2021)
- [L2CS-Net: Fine-Grained Gaze Estimation in Unconstrained Environments](https://mlsanity.com/paper/2203.03339/) (2022)
- [Multi-style Generative Network for Real-time Transfer](https://mlsanity.com/paper/1703.06953/) (2017)
- [Natural and Effective Obfuscation by Head Inpainting](https://mlsanity.com/paper/1711.09001/) (2017)
- [AnonymousNet: Natural Face De-Identification with Measurable Privacy](https://mlsanity.com/paper/1904.12620/) (2019)
- [BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generation](https://mlsanity.com/paper/2110.11728/) (2021)
- [Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation](https://mlsanity.com/paper/1908.00999/) (2019)
- [How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals](https://mlsanity.com/paper/2008.11363/) (2020)
- [Deepfakes for Medical Video De-Identification: Privacy Protection and Diagnostic Information Preservation](https://mlsanity.com/paper/2003.00813/) (2020)
- [Where Do Deep Fakes Look? Synthetic Face Detection via Gaze Tracking](https://mlsanity.com/paper/2101.01165/) (2021)
- [Practical Digital Disguises: Leveraging Face Swaps to Protect Patient Privacy](https://mlsanity.com/paper/2204.03559/) (2022)

## Built on by (most cited first, 3 of 3)

- [PrivateGaze: Preserving User Privacy in Black-box Mobile Gaze Tracking Services](https://mlsanity.com/paper/2408.00950/) (2024)
- [Towards mitigating uncann(eye)ness in face swaps via gaze-centric loss terms](https://mlsanity.com/paper/2402.03188/) (2024)
- [Towards High Fidelity Face Swapping: A Comprehensive Survey and New Benchmark](https://mlsanity.com/paper/2605.00883/) (2026)

## Abstract (arXiv)

Face swapping combines one face's identity with another face's non-appearance attributes (expression, head pose, lighting) to generate a synthetic face. This technology is rapidly improving, but falls flat when reconstructing some attributes, particularly gaze. Image-based loss metrics that consider the full face do not effectively capture the perceptually important, yet spatially small, eye regions. Improving gaze in face swaps can improve naturalness and realism, benefiting applications in entertainment, human computer interaction, and more. Improved gaze will also directly improve Deepfake detection efforts, serving as ideal training data for classifiers that rely on gaze for classification. We propose a novel loss function that leverages gaze prediction to inform the face swap model during training and compare against existing methods. We find all methods to significantly benefit gaze in resulting face swaps.