# EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters

arXiv: https://arxiv.org/abs/2503.19416 · HTML: https://mlsanity.com/paper/2503.19416/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 25 Mar 2025.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation](https://mlsanity.com/paper/2211.12194/) (2022)
- [AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis](https://mlsanity.com/paper/2103.11078/) (2021)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss](https://mlsanity.com/paper/1905.03820/) (2019)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control](https://mlsanity.com/paper/2407.03168/) (2024)
- [emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation](https://mlsanity.com/paper/2312.15185/) (2023)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [Audio-Driven Emotional Video Portraits](https://mlsanity.com/paper/2104.07452/) (2021)
- [EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model](https://mlsanity.com/paper/2205.15278/) (2022)
- [FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset](https://mlsanity.com/paper/2203.14057/) (2022)
- [Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis](https://mlsanity.com/paper/2401.08503/) (2024)
- [FunASR: A Fundamental End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2305.11013/) (2023)
- [Learning Formation of Physically-Based Face Attributes](https://mlsanity.com/paper/2004.03458/) (2020)
- [StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles](https://mlsanity.com/paper/2301.01081/) (2023)
- [One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning](https://mlsanity.com/paper/2112.02749/) (2021)
- [Everybody's Talkin': Let Me Talk as You Want](https://mlsanity.com/paper/2001.05201/) (2020)
- [Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition](https://mlsanity.com/paper/2211.12368/) (2022)
- [Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation](https://mlsanity.com/paper/2309.04946/) (2023)
- [DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering](https://mlsanity.com/paper/2201.00791/) (2022)
- [EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis](https://mlsanity.com/paper/2404.01647/) (2024)
- [A Comparison of Discrete and Soft Speech Units for Improved Voice Conversion](https://mlsanity.com/paper/2111.02392/) (2021)
- [Frame attention networks for facial expression recognition in videos](https://mlsanity.com/paper/1907.00193/) (2019)
- [Speech Driven Talking Face Generation from a Single Image and an Emotion Condition](https://mlsanity.com/paper/2008.03592/) (2020)
- [High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies](https://mlsanity.com/paper/2010.05562/) (2020)
- [Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style](https://mlsanity.com/paper/2403.06365/) (2024)

## Built on by (most cited first, 3 of 3)

- [Cross-Modal Emotion Transfer for Emotion Editing in Talking Face Video](https://mlsanity.com/paper/2604.07786/) (2026)
- [Geometry-guided Emotion Modulation for Controllable and Photorealistic Emotional Talking Face Generation](https://mlsanity.com/paper/2608.00663/) (2026)
- [Xemo-Talker: Unlock Emotions Explicitly for Audio-Driven Talking Portrait Synthesis](https://mlsanity.com/paper/2608.14700/) (2026)

## Abstract (arXiv)

Generating emotion-specific talking head videos from audio input is an important and complex challenge for human-machine interaction. However, emotion is highly abstract concept with ambiguous boundaries, and it necessitates disentangled expression parameters to generate emotionally expressive talking head videos. In this work, we present EmoHead to synthesize talking head videos via semantic expression parameters. To predict expression parameter for arbitrary audio input, we apply an audio-expression module that can be specified by an emotion tag. This module aims to enhance correlation from audio input across various emotions. Furthermore, we leverage pre-trained hyperplane to refine facial movements by probing along the vertical direction. Finally, the refined expression parameters regularize neural radiance fields and facilitate the emotion-consistent generation of talking head videos. Experimental results demonstrate that semantic expression parameters lead to better reconstruction quality and controllability.