# Unsupervised Audiovisual Synthesis via Exemplar Autoencoders

arXiv: https://arxiv.org/abs/2001.04463 · HTML: https://mlsanity.com/paper/2001.04463/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2020.

## Built on by (most cited first, 5 of 5)

- [UniFLG: Unified Facial Landmark Generator from Text or Speech](https://mlsanity.com/paper/2302.14337/) (2023)
- [Global Rhythm Style Transfer Without Text Transcriptions](https://mlsanity.com/paper/2106.08519/) (2021)
- [LISA: Localized Image Stylization with Audio via Implicit Neural Representation](https://mlsanity.com/paper/2211.11381/) (2022)
- [Video-Specific Autoencoders for Exploring, Editing and Transmitting Videos](https://mlsanity.com/paper/2103.17261/) (2021)
- [Leave No Stone Unturned: Uncovering Holistic Audio-Visual Intrinsic Coherence for Deepfake Detection](https://mlsanity.com/paper/2603.23960/) (2026)

## Abstract (arXiv)

We present an unsupervised approach that converts the input speech of any individual into audiovisual streams of potentially-infinitely many output speakers. Our approach builds on simple autoencoders that project out-of-sample data onto the distribution of the training set. We use Exemplar Autoencoders to learn the voice, stylistic prosody, and visual appearance of a specific target exemplar speech. In contrast to existing methods, the proposed approach can be easily extended to an arbitrarily large number of speakers and styles using only 3 minutes of target audio-video data, without requiring em any training data for the input speaker. To do so, we learn audiovisual bottleneck representations that capture the structured linguistic content of speech. We outperform prior approaches on both audio and video synthesis, and provide extensive qualitative analysis on our project page -- https://www.cs.cmu.edu/~exemplar-ae/.