Who is a Better Talker: Subjective and Objective Quality Assessment for AI-Generated Talking Heads
Not on the map. Why?
Abstract (arXiv)
Speech-driven methods for portraits are figuratively known as "Talkers" because of their capability to synthesize speaking mouth shapes and facial movements. Especially with the rapid development of the Text-to-Image (T2I) models, AI-Generated Talking Heads (AGTHs) have gradually become an emerging digital human media. However, challenges persist regarding the quality of these talkers and AGTHs they generate, and comprehensive studies addressing these issues remain limited. To address this gap, this paper presents the largest AGTH quality assessment dataset THQA-10K to date, which selects 12 prominent T2I models and 14 advanced talkers to generate AGTHs for 14 prompts. After excluding instances where AGTH generation is unsuccessful, the THQA-10K dataset contains 10,457 AGTHs. Then, volunteers are recruited to subjectively rate the AGTHs and give the corresponding distortion categories. In our analysis for subjective experimental results, we evaluate the performance of talkers in terms of generalizability and quality, and also expose the distortions of existing AGTHs. Finally, an objective quality assessment method based on the first frame, Y-T slice and tone-lip consistency is proposed. Experimental results show that this method can achieve state-of-the-art (SOTA) performance in AGTH quality assessment. The work is released at https://github.com/zyj-2000/Talker.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 76 references that are in the map. First posted on arXiv 31 Jul 2025.
Builds on
41 of 76 references.
References · 41
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
- 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
- 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 178
- 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
29 more
- 2019Quality Assessment of In-the-Wild Videoscited by 164
- 2021Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motioncited by 149
- 2020UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Contentcited by 140
- 2022A Deep Learning based No-reference Quality Assessment Model for UGC Videoscited by 129
- 2022VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wildcited by 106
- 2021RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Contentcited by 100
- 2023Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expertcited by 98
- 2023Identity-Preserving Talking Face Generation with Landmark and Appearance Priorscited by 98
- 2021Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perceptioncited by 97
- 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
- 2023DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Videocited by 74
- 2024Perceptual Video Quality Assessment: A Surveycited by 43
- 2024A-Bench: Are LMMs Masters at Evaluating AI-generated Images?cited by 39
- 2024EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Modelscited by 37
- 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
- 2023Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluationcited by 27
- 2024Quality Assessment in the Era of Large Models: A Surveycited by 26
- 2025Q-Agent: Quality-Driven Chain-of-Thought Image Restoration Agent through Robust Multimodal Large Language Modelcited by 15
- 2022Perceptual Quality Assessment for Digital Human Headscited by 13
- 2024MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysiscited by 11
- 2022DDH-QA: A Dynamic Digital Humans Quality Assessment Databasecited by 9
- 2023EEP-3DQA: Efficient and Effective Projection-based 3D Model Quality Assessmentcited by 8
- 2024THQA: A Perceptual Quality Assessment Database for Talking Headscited by 8
- 2023Geometry-Aware Video Quality Assessment for Dynamic Digital Humancited by 7
- 2023A No-Reference Quality Assessment Method for Digital Human Headcited by 6
- 2023An Implementation of Multimodal Fusion System for Intelligent Digital Human Generationcited by 6
- 20243DGCQA: A Quality Assessment Database for 3D AI-Generated Contentscited by 6
- 2025NTIRE 2025 XGC Quality Assessment Challenge: Methods and Resultscited by 6
- 2023Simple Baselines for Projection-based Full-reference and No-reference Point Cloud Quality Assessmentcited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.