# Speech2Face: Learning the Face Behind a Voice

arXiv: https://arxiv.org/abs/1905.09773 · HTML: https://mlsanity.com/paper/1905.09773/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2019.

## Built on by (most cited first, 30 of 55)

- [VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency](https://mlsanity.com/paper/2101.03149/) (2021)
- [Voice-Face Homogeneity Tells Deepfake](https://mlsanity.com/paper/2203.02195/) (2022)
- [Sound to Visual Scene Generation by Audio-to-Visual Latent Alignment](https://mlsanity.com/paper/2303.17490/) (2023)
- [Sound-Guided Semantic Image Manipulation](https://mlsanity.com/paper/2112.00007/) (2021)
- [Deep Learning-based Face Super-Resolution: A Survey](https://mlsanity.com/paper/2101.03749/) (2021)
- [Seeking the Shape of Sound: An Adaptive Framework for Learning Voice-Face Association](https://mlsanity.com/paper/2103.07293/) (2021)
- [FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audio](https://mlsanity.com/paper/2403.01901/) (2024)
- [Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs](https://mlsanity.com/paper/2105.12049/) (2021)
- [AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person](https://mlsanity.com/paper/2108.04325/) (2021)
- [Strumming to the Beat: Audio-Conditioned Contrastive Video Textures](https://mlsanity.com/paper/2104.02687/) (2021)
- [Finding Fallen Objects Via Asynchronous Audio-Visual Integration](https://mlsanity.com/paper/2207.03483/) (2022)
- [A cappella: Audio-visual Singing Voice Separation](https://mlsanity.com/paper/2104.09946/) (2021)
- [Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Control](https://mlsanity.com/paper/2503.14517/) (2025)
- [Residual-guided Personalized Speech Synthesis based on Face Image](https://mlsanity.com/paper/2204.01672/) (2022)
- [Speech2Video: Cross-Modal Distillation for Speech to Video Generation](https://mlsanity.com/paper/2107.04806/) (2021)
- [TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentation](https://mlsanity.com/paper/2307.14611/) (2023)
- [AVI-Edit: Audio-sync Video Instance Editing with Granularity-Aware Mask Refiner](https://mlsanity.com/paper/2512.10571/) (2025)
- [Cross-Modal Perceptionist: Can Face Geometry be Gleaned from Voices?](https://mlsanity.com/paper/2203.09824/) (2022)
- [Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice Alignment](https://mlsanity.com/paper/2309.09470/) (2023)
- [Robust Sound-Guided Image Manipulation](https://mlsanity.com/paper/2208.14114/) (2022)
- [Collaborative Learning to Generate Audio-Video Jointly](https://mlsanity.com/paper/2104.02656/) (2021)
- [Conditional Vector Graphics Generation for Music Cover Images](https://mlsanity.com/paper/2205.07301/) (2022)
- [Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignment](https://mlsanity.com/paper/2412.06209/) (2024)
- [Template-Free Try-on Image Synthesis via Semantic-guided Optimization](https://mlsanity.com/paper/2102.03503/) (2021)
- [SoK: A Modularized Approach to Study the Security of Automatic Speech Recognition Systems](https://mlsanity.com/paper/2103.10651/) (2021)
- [Facetron: A Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representations](https://mlsanity.com/paper/2107.12003/) (2021)
- [FaVoA: Face-Voice Association Favours Ambiguous Speaker Detection](https://mlsanity.com/paper/2109.00577/) (2021)
- [Hear Your Face: Face-based voice conversion with F0 estimation](https://mlsanity.com/paper/2408.09802/) (2024)
- [Voice2Mesh: Cross-Modal 3D Face Model Generation from Voices](https://mlsanity.com/paper/2104.10299/) (2021)
- [The Boombox: Visual Reconstruction from Acoustic Vibrations](https://mlsanity.com/paper/2105.08052/) (2021)

## Abstract (arXiv)

How much can we infer about a person's looks from the way they speak? In this paper, we study the task of reconstructing a facial image of a person from a short audio recording of that person speaking. We design and train a deep neural network to perform this task using millions of natural Internet/YouTube videos of people speaking. During training, our model learns voice-face correlations that allow it to produce images that capture various physical attributes of the speakers such as age, gender and ethnicity. This is done in a self-supervised manner, by utilizing the natural co-occurrence of faces and speech in Internet videos, without the need to model attributes explicitly. We evaluate and numerically quantify how--and in what manner--our Speech2Face reconstructions, obtained directly from audio, resemble the true face images of the speakers.