mlsanityData to 7 Oct 2026

Speech2Face: Learning the Face Behind a Voice

arXiv 1905.09773 ↗First posted 23 May 2019cs.CV cs.MMcited by 55

Not on the map. Why?

Abstract (arXiv)

How much can we infer about a person's looks from the way they speak? In this paper, we study the task of reconstructing a facial image of a person from a short audio recording of that person speaking. We design and train a deep neural network to perform this task using millions of natural Internet/YouTube videos of people speaking. During training, our model learns voice-face correlations that allow it to produce images that capture various physical attributes of the speakers such as age, gender and ethnicity. This is done in a self-supervised manner, by utilizing the natural co-occurrence of faces and speech in Internet videos, without the need to model attributes explicitly. We evaluate and numerically quantify how--and in what manner--our Speech2Face reconstructions, obtained directly from audio, resemble the true face images of the speakers.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 20; 2022: 12; 2023: 10; 2024: 7; 2025: 6.

55 papers.

Sort:

By area:

  1. 2021VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistencycited by 114
  2. 2022Voice-Face Homogeneity Tells Deepfakecited by 44
  3. 2023Sound to Visual Scene Generation by Audio-to-Visual Latent Alignmentcited by 41
  4. 2021Sound-Guided Semantic Image Manipulationcited by 35
  5. 2021Deep Learning-based Face Super-Resolution: A Surveycited by 32
  6. 2021Seeking the Shape of Sound: An Adaptive Framework for Learning Voice-Face Associationcited by 21
  7. 2024FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audiocited by 20
  8. 2021Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputscited by 18
  9. 2021AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Personcited by 17
  10. 2021Strumming to the Beat: Audio-Conditioned Contrastive Video Texturescited by 14
  11. 2022Finding Fallen Objects Via Asynchronous Audio-Visual Integrationcited by 13
  12. 2021A cappella: Audio-visual Singing Voice Separationcited by 12
38 more
  1. 2025Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Controlcited by 12
  2. 2022Residual-guided Personalized Speech Synthesis based on Face Imagecited by 10
  3. 2021Speech2Video: Cross-Modal Distillation for Speech to Video Generationcited by 9
  4. 2023TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentationcited by 9
  5. 2022Cross-Modal Perceptionist: Can Face Geometry be Gleaned from Voices?cited by 8
  6. 2023Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice Alignmentcited by 8
  7. 2025AVI-Edit: Audio-sync Video Instance Editing with Granularity-Aware Mask Refinercited by 8
  8. 2022Robust Sound-Guided Image Manipulationcited by 7
  9. 2021Collaborative Learning to Generate Audio-Video Jointlycited by 6
  10. 2022Conditional Vector Graphics Generation for Music Cover Imagescited by 5
  11. 2024Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignmentcited by 5
  12. 2021Template-Free Try-on Image Synthesis via Semantic-guided Optimizationcited by 4
  13. 2021SoK: A Modularized Approach to Study the Security of Automatic Speech Recognition Systemscited by 4
  14. 2021Facetron: A Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representationscited by 4
  15. 2021FaVoA: Face-Voice Association Favours Ambiguous Speaker Detectioncited by 4
  16. 2024Hear Your Face: Face-based voice conversion with F0 estimationcited by 4
  17. 2021Voice2Mesh: Cross-Modal 3D Face Model Generation from Voicescited by 3
  18. 2021The Boombox: Visual Reconstruction from Acoustic Vibrationscited by 3
  19. 2021Controlled AutoEncoders to Generate Faces from Voicescited by 3
  20. 2023Rethinking Voice-Face Correlation: A Geometry Viewcited by 3
  21. 2023A Survey on Deep Multi-modal Learning for Body Language Recognition and Generationcited by 3
  22. 2024Misrepresented Technological Solutions in Imagined Futures: The Origins and Dangers of AI Hype in the Research Communitycited by 3
  23. 2024Seeing Your Speech Style: A Novel Zero-Shot Identity-Disentanglement Face-based Voice Conversioncited by 3
  24. 2025Advances in Set Function Learning: A Survey of Techniques and Applicationscited by 3
  25. 2021Improving Ultrasound Tongue Image Reconstruction from Lip Images Using Self-supervised Learning and Attention Mechanismcited by 2
  26. 2021Multimodal perception for dexterous manipulationcited by 2
  27. 2022Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detectioncited by 2
  28. 2023Fusion-S2iGan: An Efficient and Effective Single-Stage Framework for Speech-to-Image Generationcited by 2
  29. 2025MACS: Multi-source Audio-to-image Generation with Contextual Significance and Semantic Alignmentcited by 2
  30. 2021Cross-Modal Virtual Sensing for Combustion Instability Monitoringcited by 1
  31. 2022Triangular Character Animation Sampling with Motion, Emotion, and Relationcited by 1
  32. 2022Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priorscited by 1
  33. 2023Cross-modal Face- and Voice-style Transfercited by 1
  34. 2023Looking and Listening: Audio Guided Text Recognitioncited by 1
  35. 2023Realistic Speech-to-Face Generation with Speech-Conditioned Latent Diffusion Model with Face Priorcited by 1
  36. 2024Qualitative Approaches to Voice UXcited by 1
  37. 2024Enhancing Human Action Recognition and Violence Detection Through Deep Learning Audiovisual Fusioncited by 1
  38. 2022Show Me Your Face, And I'll Tell You How You Speakcited by 0

50 most cited of 55 All 55 (JSON)