mlsanityData to 14 Oct 2026

Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generation

arXiv 2507.12761 ↗First posted 17 Jul 2025cs.CV cs.AIcited by 0

On the map: LLM reasoning → Audio-language model reasoning

also in 1 more
Abstract (arXiv)

Emotional talking-head generation has emerged as a pivotal research area at the intersection of computer vision and multimodal artificial intelligence, with its core value lying in enhancing human-computer interaction through immersive and empathetic engagement.With the advancement of multimodal large language models, the driving signals for emotional talking-head generation has shifted from audio and video to more flexible text. However, current text-driven methods rely on predefined discrete emotion label texts, oversimplifying the dynamic complexity of real facial muscle movements and thus failing to achieve natural emotional expressiveness.This study proposes the Think-Before-Draw framework to address two key challenges: (1) In-depth semantic parsing of emotions--by innovatively introducing Chain-of-Thought (CoT), abstract emotion labels are transformed into physiologically grounded facial muscle movement descriptions, enabling the mapping from high-level semantics to actionable motion features; and (2) Fine-grained expressiveness optimization--inspired by artists' portrait painting process, a progressive guidance denoising strategy is proposed, employing a "global emotion localization--local muscle control" mechanism to refine micro-expression dynamics in generated videos.Our experiments demonstrate that our approach achieves state-of-the-art performance on widely-used benchmarks, including MEAD and HDTF. Additionally, we collected a set of portrait images to evaluate our model's zero-shot generation capability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 14 of its 60 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

14 of 60 references.

References · 14

  1. 2020Denoising Diffusion Implicit Modelscited by 9,781
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  6. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  7. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  8. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  9. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 202
  10. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
  11. 2024Sonic: Shifting Focus to Global Audio Perception in Portrait Animationcited by 72
  12. 2024A Survey on Generative AI and LLM for Video Generation, Understanding, and Streamingcited by 37
2 more
  1. 2025MoEE: Mixture of Emotion Experts for Audio-Driven Portrait Animationcited by 12
  2. 2019wav2vec: Unsupervised Pre-training for Speech Recognitioncited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion
  2. 2024EMOdiffhead: Continuously Emotional Control in Talking Head Generation via Diffusion
  3. 2026MindFlow: Harmonizing Cognitive Semantics and Acoustic Dynamics for Facial Animation Generation in Dyadic Conversations
  4. 2024Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style
  5. 2026AUHead: Realistic Emotional Talking Head Generation via Action Units Control
  6. 2025Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation
  7. 2025Controllable Expressive 3D Facial Animation via Diffusion in a Unified Multimodal Space
  8. 2025X-Actor: Emotional and Expressive Long-Range Portrait Acting from Audio