# LaughTalk: Expressive 3D Talking Head Generation with Laughter

arXiv: https://arxiv.org/abs/2311.00994 · HTML: https://mlsanity.com/paper/2311.00994/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 2 Nov 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Deep Audio-Visual Speech Recognition](https://mlsanity.com/paper/1809.02108/) (2018)
- [Capture, Learning, and Synthesis of 3D Speaking Styles](https://mlsanity.com/paper/1905.03079/) (2019)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement](https://mlsanity.com/paper/2104.08223/) (2021)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection](https://mlsanity.com/paper/2107.06592/) (2021)
- [CelebV-Text: A Large-Scale Facial Text-Video Dataset](https://mlsanity.com/paper/2303.14717/) (2023)
- [Emotional Speech-Driven Animation with Content-Emotion Disentanglement](https://mlsanity.com/paper/2306.08990/) (2023)
- [Multiface: A Dataset for Neural Face Rendering](https://mlsanity.com/paper/2207.11243/) (2022)
- [Imitator: Personalized Speech-driven 3D Facial Animation](https://mlsanity.com/paper/2301.00023/) (2022)
- [Modality Dropout for Improved Performance-driven Talking Faces](https://mlsanity.com/paper/2005.13616/) (2020)
- [Audio2Face: Generating Speech/Face Animation from Single Audio with Attention-Based Bidirectional LSTM Networks](https://mlsanity.com/paper/1905.11142/) (2019)
- [A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization](https://mlsanity.com/paper/2310.03205/) (2023)

## Built on by (most cited first, 27 of 27)

- [Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering](https://mlsanity.com/paper/2312.11360/) (2023)
- [Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion](https://mlsanity.com/paper/2312.04466/) (2023)
- [DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations](https://mlsanity.com/paper/2505.18096/) (2025)
- [SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models](https://mlsanity.com/paper/2312.09818/) (2023)
- [MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset](https://mlsanity.com/paper/2406.14272/) (2024)
- [EmoVOCA: Speech-Driven Emotional 3D Talking Heads](https://mlsanity.com/paper/2403.12886/) (2024)
- [DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation](https://mlsanity.com/paper/2408.06010/) (2024)
- [KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation](https://mlsanity.com/paper/2503.01715/) (2025)
- [VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models](https://mlsanity.com/paper/2504.02386/) (2025)
- [UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speaking](https://mlsanity.com/paper/2512.09327/) (2025)
- [Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics](https://mlsanity.com/paper/2503.20308/) (2025)
- [Enhancing Speech-Driven 3D Facial Animation with Audio-Visual Guidance from Lip Reading Expert](https://mlsanity.com/paper/2407.01034/) (2024)
- [Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions](https://mlsanity.com/paper/2507.02900/) (2025)
- [A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing](https://mlsanity.com/paper/2406.10553/) (2024)
- [Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignment](https://mlsanity.com/paper/2412.06209/) (2024)
- [3DiFACE: Synthesizing and Editing Holistic 3D Facial Animation](https://mlsanity.com/paper/2509.26233/) (2025)
- [JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Model](https://mlsanity.com/paper/2408.01627/) (2024)
- [Towards Seamless Interaction: Causal Turn-Level Modeling of Interactive 3D Conversational Head Dynamics](https://mlsanity.com/paper/2512.15340/) (2025)
- [Diverse Code Query Learning for Speech-Driven Facial Animation](https://mlsanity.com/paper/2409.19143/) (2024)
- [When Words Smile: Generating Diverse Emotional Facial Expressions from Text](https://mlsanity.com/paper/2412.02508/) (2024)
- [Tiny is not small enough: High-quality, low-resource facial animation models through hybrid knowledge distillation](https://mlsanity.com/paper/2507.18352/) (2025)
- [EmoFace: Emotion-Content Disentangled Speech-Driven 3D Talking Face Animation](https://mlsanity.com/paper/2408.11518/) (2024)
- [EmoGene: Audio-Driven Emotional 3D Talking-Head Generation](https://mlsanity.com/paper/2410.17262/) (2024)
- [RealTalk: Realistic Emotion-Aware Lifelike Talking-Head Synthesis](https://mlsanity.com/paper/2508.12163/) (2025)
- [Learning Disentangled Speech- and Expression-Driven Blendshapes for 3D Talking Face Animation](https://mlsanity.com/paper/2510.25234/) (2025)
- [3DXTalker: Unifying Identity, Lip Sync, Emotion, and Spatial Dynamics in Expressive 3D Talking Avatars](https://mlsanity.com/paper/2602.10516/) (2026)
- [Talking Together: Synthesizing Co-Located 3D Conversations from Audio](https://mlsanity.com/paper/2603.08674/) (2026)

## Abstract (arXiv)

Laughter is a unique expression, essential to affirmative social interactions of humans. Although current 3D talking head generation methods produce convincing verbal articulations, they often fail to capture the vitality and subtleties of laughter and smiles despite their importance in social context. In this paper, we introduce a novel task to generate 3D talking heads capable of both articulate speech and authentic laughter. Our newly curated dataset comprises 2D laughing videos paired with pseudo-annotated and human-validated 3D FLAME parameters and vertices. Given our proposed dataset, we present a strong baseline with a two-stage training scheme: the model first learns to talk and then acquires the ability to express laughter. Extensive experiments demonstrate that our method performs favorably compared to existing approaches in both talking head generation and expressing laughter signals. We further explore potential applications on top of our proposed method for rigging realistic avatars.