# Face Photo-Sketch Recognition Using Bidirectional Collaborative Synthesis Network

arXiv: https://arxiv.org/abs/2108.09898 · HTML: https://mlsanity.com/paper/2108.09898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 23 Aug 2021.

## Builds on (most cited first, 10 of 10)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations](https://mlsanity.com/paper/1905.00292/) (2019)
- [Learning to Generate Images with Perceptual Similarity Metrics](https://mlsanity.com/paper/1511.06409/) (2015)
- [Graphical Representation for Heterogeneous Face Recognition](https://mlsanity.com/paper/1503.00488/) (2015)
- [Attribute-Centered Loss for Soft-Biometrics Guided Face Sketch-Photo Recognition](https://mlsanity.com/paper/1804.03082/) (2018)
- [Deep Sketch-Photo Face Recognition Assisted by Facial Attributes](https://mlsanity.com/paper/1808.00059/) (2018)

## Abstract (arXiv)

This research features a deep-learning based framework to address the problem of matching a given face sketch image against a face photo database. The problem of photo-sketch matching is challenging because 1) there is large modality gap between photo and sketch, and 2) the number of paired training samples is insufficient to train deep learning based networks. To circumvent the problem of large modality gap, our approach is to use an intermediate latent space between the two modalities. We effectively align the distributions of the two modalities in this latent space by employing a bidirectional (photo -> sketch and sketch -> photo) collaborative synthesis network. A StyleGAN-like architecture is utilized to make the intermediate latent space be equipped with rich representation power. To resolve the problem of insufficient training samples, we introduce a three-step training scheme. Extensive evaluation on public composite face sketch database confirms superior performance of our method compared to existing state-of-the-art methods. The proposed methodology can be employed in matching other modality pairs.