# Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition

arXiv: https://arxiv.org/abs/2103.16019 · HTML: https://mlsanity.com/paper/2103.16019/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [InstaGAN: Instance-aware Image-to-Image Translation](https://mlsanity.com/paper/1812.10889/) (2018)
- [End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning](https://mlsanity.com/paper/1501.07180/) (2015)
- [High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks](https://mlsanity.com/paper/1710.10182/) (2017)
- [Unsupervised Facial Geometry Learning for Sketch to Photo Synthesis](https://mlsanity.com/paper/1810.05361/) (2018)
- [Face Translation between Images and Videos using Identity-aware CycleGAN](https://mlsanity.com/paper/1712.00971/) (2017)

## Built on by (most cited first, 30 of 39)

- [Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment](https://mlsanity.com/paper/2203.04121/) (2022)
- [Scientific and Technological Information Oriented Semantics-adversarial and Media-adversarial Cross-media Retrieval](https://mlsanity.com/paper/2203.08615/) (2022)
- [Prepended Domain Transformer: Heterogeneous Face Recognition without Bells and Whistles](https://mlsanity.com/paper/2210.06529/) (2022)
- [Facial-Sketch Synthesis: A New Challenge](https://mlsanity.com/paper/2112.15439/) (2021)
- [3D Pose Estimation and Future Motion Prediction from 2D Images](https://mlsanity.com/paper/2111.13285/) (2021)
- [Bridging the Gap: Heterogeneous Face Recognition with Conditional Adaptive Instance Modulation](https://mlsanity.com/paper/2307.07032/) (2023)
- [AdvCloak: Customized Adversarial Cloak for Privacy Protection](https://mlsanity.com/paper/2312.14407/) (2023)
- [xEdgeFace: Efficient Cross-Spectral Face Recognition for Edge Devices](https://mlsanity.com/paper/2504.19646/) (2025)
- [Sketch Less Face Image Retrieval: A New Challenge](https://mlsanity.com/paper/2302.05576/) (2023)
- [Modality Agnostic Heterogeneous Face Recognition with Switch Style Modulators](https://mlsanity.com/paper/2407.08640/) (2024)
- [A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets](https://mlsanity.com/paper/2107.07927/) (2021)
- [Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals](https://mlsanity.com/paper/2203.05864/) (2022)
- [From Modalities to Styles: Rethinking the Domain Gap in Heterogeneous Face Recognition](https://mlsanity.com/paper/2404.14247/) (2024)
- [Semi-supervised Cycle-GAN for face photo-sketch translation in the wild](https://mlsanity.com/paper/2307.10281/) (2023)
- [Semantic Similarity Computing for Scientific Academic Conferences fused with domain features](https://mlsanity.com/paper/2203.12593/) (2022)
- [Cross-modal Generative Model for Visual-Guided Binaural Stereo Generation](https://mlsanity.com/paper/2311.07630/) (2023)
- [SPOT-Face: Forensic Face Identification using Attention Guided Optimal Transport](https://mlsanity.com/paper/2601.09229/) (2026)
- [Lightweight Cross-Spectral Face Recognition via Contrastive Alignment and Distillation](https://mlsanity.com/paper/2605.04769/) (2026)
- [Face sketch to photo translation using generative adversarial networks](https://mlsanity.com/paper/2110.12290/) (2021)
- [An Intellectual Property Entity Recognition Method Based on Transformer and Technological Word Information](https://mlsanity.com/paper/2203.10717/) (2022)
- [Prediction Algorithm for Heat Demand of Science and Technology Topics Based on Time Convolution Network](https://mlsanity.com/paper/2203.10718/) (2022)
- [Research Scholar Interest Mining Method based on Load Centrality](https://mlsanity.com/paper/2203.10731/) (2022)
- [Academic Resource Text Level Multi-label Classification based on Attention](https://mlsanity.com/paper/2203.10743/) (2022)
- [Cross-Media Scientific Research Achievements Retrieval Based on Deep Language Model](https://mlsanity.com/paper/2203.15595/) (2022)
- [Research topic trend prediction of scientific papers based on spatial enhancement and dynamic graph convolution network](https://mlsanity.com/paper/2203.16256/) (2022)
- [An unsupervised cluster-level based method for learning node representations of heterogeneous graphs in scientific papers](https://mlsanity.com/paper/2203.16751/) (2022)
- [Scientific and Technological Text Knowledge Extraction Method of based on Word Mixing and GRU](https://mlsanity.com/paper/2203.17079/) (2022)
- [Accurate Portraits of Scientific Resources and Knowledge Service Components](https://mlsanity.com/paper/2204.04883/) (2022)
- [Research on Cross-media Science and Technology Information Data Retrieval](https://mlsanity.com/paper/2204.04887/) (2022)
- [Knowledge Graph and Accurate Portrait Construction of Scientific and Technological Academic Conferences](https://mlsanity.com/paper/2204.04888/) (2022)

## Abstract (arXiv)

Face photo-sketch synthesis and recognition has many applications in digital entertainment and law enforcement. Recently, generative adversarial networks (GANs) based methods have significantly improved the quality of image synthesis, but they have not explicitly considered the purpose of recognition. In this paper, we first propose an Identity-Aware CycleGAN (IACycleGAN) model that applies a new perceptual loss to supervise the image generation network. It improves CycleGAN on photo-sketch synthesis by paying more attention to the synthesis of key facial regions, such as eyes and nose, which are important for identity recognition. Furthermore, we develop a mutual optimization procedure between the synthesis model and the recognition model, which iteratively synthesizes better images by IACycleGAN and enhances the recognition model by the triplet loss of the generated and real samples. Extensive experiments are performed on both photo-tosketch and sketch-to-photo tasks using the widely used CUFS and CUFSF databases. The results show that the proposed method performs better than several state-of-the-art methods in terms of both synthetic image quality and photo-sketch recognition accuracy.