# ObamaNet: Photo-realistic lip-sync from text

arXiv: https://arxiv.org/abs/1801.01442 · HTML: https://mlsanity.com/paper/1801.01442/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2017.

## Built on by (most cited first, 30 of 42)

- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [FT2TF: First-Person Statement Text-To-Talking Face Generation](https://mlsanity.com/paper/2312.05430/) (2023)
- [Deep Learning for Visual Speech Analysis: A Survey](https://mlsanity.com/paper/2205.10839/) (2022)
- [Faces that Speak: Jointly Synthesising Talking Face and Speech from Text](https://mlsanity.com/paper/2405.10272/) (2024)
- [Face-Dubbing++: Lip-Synchronous, Voice Preserving Translation of Videos](https://mlsanity.com/paper/2206.04523/) (2022)
- [AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person](https://mlsanity.com/paper/2108.04325/) (2021)
- [Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis](https://mlsanity.com/paper/2109.02081/) (2021)
- [More than Words: In-the-Wild Visually-Driven Prosody for Text-to-Speech](https://mlsanity.com/paper/2111.10139/) (2021)
- [Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video](https://mlsanity.com/paper/2309.04814/) (2023)
- [MoDiTalker: Motion-Disentangled Diffusion Model for High-Fidelity Talking Head Generation](https://mlsanity.com/paper/2403.19144/) (2024)
- [AV-Flow: Transforming Text to Audio-Visual Human-like Interactions](https://mlsanity.com/paper/2502.13133/) (2025)
- [OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimicking](https://mlsanity.com/paper/2504.02433/) (2025)
- [Lip-to-Speech Synthesis for Arbitrary Speakers in the Wild](https://mlsanity.com/paper/2209.00642/) (2022)
- [UniFLG: Unified Facial Landmark Generator from Text or Speech](https://mlsanity.com/paper/2302.14337/) (2023)
- [Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generation](https://mlsanity.com/paper/2308.06457/) (2023)
- [Parallel and High-Fidelity Text-to-Lip Generation](https://mlsanity.com/paper/2107.06831/) (2021)
- [Attention-Based Lip Audio-Visual Synthesis for Talking Face Generation in the Wild](https://mlsanity.com/paper/2203.03984/) (2022)
- [VAST: Vivify Your Talking Avatar via Zero-Shot Expressive Facial Style Transfer](https://mlsanity.com/paper/2308.04830/) (2023)
- [Neural Text to Articulate Talk: Deep Text to Audiovisual Speech Synthesis achieving both Auditory and Photo-realism](https://mlsanity.com/paper/2312.06613/) (2023)
- [The Lost Melody: Empirical Observations on Text-to-Video Generation From A Storytelling Perspective](https://mlsanity.com/paper/2405.08720/) (2024)
- [An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detection](https://mlsanity.com/paper/2203.05178/) (2022)
- [End to End Lip Synchronization with a Temporal AutoEncoder](https://mlsanity.com/paper/2203.16224/) (2022)
- [Towards MOOCs for Lipreading: Using Synthetic Talking Heads to Train Humans in Lipreading at Scale](https://mlsanity.com/paper/2208.09796/) (2022)
- [Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection](https://mlsanity.com/paper/2212.05667/) (2022)
- [HDTR-Net: A Real-Time High-Definition Teeth Restoration Network for Arbitrary Talking Face Generation Methods](https://mlsanity.com/paper/2309.07495/) (2023)
- [MMGT: Motion Mask Guided Two-Stage Network for Co-Speech Gesture Video Generation](https://mlsanity.com/paper/2505.23120/) (2025)
- [Controllable and Expressive One-Shot Video Head Swapping](https://mlsanity.com/paper/2506.16852/) (2025)
- [ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splatting](https://mlsanity.com/paper/2601.01847/) (2026)
- [Fake Visual Content Detection Using Two-Stream Convolutional Neural Networks](https://mlsanity.com/paper/2101.00676/) (2021)
- [Fine-grained Identity Preserving Landmark Synthesis for Face Reenactment](https://mlsanity.com/paper/2110.04708/) (2021)

## Abstract (arXiv)

We present ObamaNet, the first architecture that generates both audio and synchronized photo-realistic lip-sync videos from any new text. Contrary to other published lip-sync approaches, ours is only composed of fully trainable neural modules and does not rely on any traditional computer graphics methods. More precisely, we use three main modules: a text-to-speech network based on Char2Wav, a time-delayed LSTM to generate mouth-keypoints synced to the audio, and a network based on Pix2Pix to generate the video frames conditioned on the keypoints.