mlsanityData to 14 Oct 2026

A Comparative Study of Perceptual Quality Metrics for Audio-driven Talking Head Videos

arXiv 2403.06421 ↗First posted 11 Mar 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

The rapid advancement of Artificial Intelligence Generated Content (AIGC) technology has propelled audio-driven talking head generation, gaining considerable research attention for practical applications. However, performance evaluation research lags behind the development of talking head generation techniques. Existing literature relies on heuristic quantitative metrics without human validation, hindering accurate progress assessment. To address this gap, we collect talking head videos generated from four generative methods and conduct controlled psychophysical experiments on visual quality, lip-audio synchronization, and head movement naturalness. Our experiments validate consistency between model predictions and human annotations, identifying metrics that align better with human opinions than widely-used measures. We believe our work will facilitate performance evaluation and model development, providing insights into AIGC in a broader context. Code and data will be made available at https://github.com/zwx8981/ADTH-QA.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

18 of 27 references.

References · 18

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  3. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
  4. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  5. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  6. 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
  7. 2019Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Networkcited by 302
  8. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 280
  9. 2018Talking Face Generation by Adversarially Disentangled Audio-Visual Representationcited by 212
  10. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  11. 2018Lip Movements Generation at a Glancecited by 155
  12. 2020Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wildcited by 134
6 more
  1. 2021Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perceptioncited by 97
  2. 2022DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Renderingcited by 80
  3. 2020Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Trainingcited by 72
  4. 2019ViSiL: Fine-grained Spatio-Temporal Video Similarity Learningcited by 42
  5. 2022Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectorscited by 26
  6. 2020A Comparative Evaluation of Temporal Pooling Methods for Blind Video Quality Assessmentcited by 22

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Dual-Branch Network for Portrait Image Quality Assessmentcited by 8
  2. 2025THEval. Evaluation Framework for Talking Head Video Generationcited by 3
  3. 2024LES-Talker: Fine-Grained Emotion Editing for Talking Head Generation in Linear Emotion Spacecited by 1
  4. 2025AI killed the video star. Audio-driven diffusion model for expressive talking head generationcited by 0