# What makes you, you? Analyzing Recognition by Swapping Face Parts

arXiv: https://arxiv.org/abs/2206.11759 · HTML: https://mlsanity.com/paper/2206.11759/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [On Face Segmentation, Face Swapping, and Face Perception](https://mlsanity.com/paper/1704.06729/) (2017)
- [FSNet: An Identity-Aware Generative Model for Image-based Face Swapping](https://mlsanity.com/paper/1811.12666/) (2018)

## Built on by (most cited first, 4 of 4)

- [Automatic Generation of Semantic Parts for Face Image Synthesis](https://mlsanity.com/paper/2307.05317/) (2023)
- [FuseAnyPart: Diffusion-Driven Facial Parts Swapping via Multiple Reference Images](https://mlsanity.com/paper/2410.22771/) (2024)
- [Adversarial Identity Injection for Semantic Face Image Synthesis](https://mlsanity.com/paper/2404.10408/) (2024)
- [Modelship Attribution: Tracing Multi-Stage Manipulations Across Generative Models](https://mlsanity.com/paper/2506.02405/) (2025)

## Abstract (arXiv)

Deep learning advanced face recognition to an unprecedented accuracy. However, understanding how local parts of the face affect the overall recognition performance is still mostly unclear. Among others, face swap has been experimented to this end, but just for the entire face. In this paper, we propose to swap facial parts as a way to disentangle the recognition relevance of different face parts, like eyes, nose and mouth. In our method, swapping parts from a source face to a target one is performed by fitting a 3D prior, which establishes dense pixels correspondence between parts, while also handling pose differences. Seamless cloning is then used to obtain smooth transitions between the mapped source regions and the shape and skin tone of the target face. We devised an experimental protocol that allowed us to draw some preliminary conclusions when the swapped images are classified by deep networks, indicating a prominence of the eyes and eyebrows region. Code available at https://github.com/clferrari/FacePartsSwap