# EvalTalker: Learning to Evaluate Real-Portrait-Driven Multi-Subject Talking Humans

arXiv: https://arxiv.org/abs/2512.01340 · HTML: https://mlsanity.com/paper/2512.01340/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 70 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Builds on (most cited first, 30 of 36)

- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation](https://mlsanity.com/paper/2211.12194/) (2022)
- [MakeItTalk: Speaker-Aware Talking-Head Animation](https://mlsanity.com/paper/2004.12992/) (2020)
- [Sapiens: Foundation for Human Vision Models](https://mlsanity.com/paper/2408.12569/) (2024)
- [FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling](https://mlsanity.com/paper/2207.02595/) (2022)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion](https://mlsanity.com/paper/2107.09293/) (2021)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [A Deep Learning based No-reference Quality Assessment Model for UGC Videos](https://mlsanity.com/paper/2204.14047/) (2022)
- [Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer](https://mlsanity.com/paper/2412.00733/) (2024)
- [VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild](https://mlsanity.com/paper/2211.14758/) (2022)
- [RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content](https://mlsanity.com/paper/2101.10955/) (2021)
- [Identity-Preserving Talking Face Generation with Landmark and Appearance Priors](https://mlsanity.com/paper/2305.08293/) (2023)
- [Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perception](https://mlsanity.com/paper/2108.08505/) (2021)
- [OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation](https://mlsanity.com/paper/2506.18866/) (2025)
- [DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video](https://mlsanity.com/paper/2303.03988/) (2023)
- [Sonic: Shifting Focus to Global Audio Perception in Portrait Animation](https://mlsanity.com/paper/2411.16331/) (2024)
- [HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characters](https://mlsanity.com/paper/2505.20156/) (2025)
- [AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding](https://mlsanity.com/paper/2405.03121/) (2024)
- [Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation](https://mlsanity.com/paper/2505.22647/) (2025)
- [LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision](https://mlsanity.com/paper/2412.09262/) (2024)
- [A-Bench: Are LMMs Masters at Evaluating AI-generated Images?](https://mlsanity.com/paper/2406.03070/) (2024)
- [Generative Artificial Intelligence: A Systematic Review and Applications](https://mlsanity.com/paper/2405.11029/) (2024)
- [Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluation](https://mlsanity.com/paper/2307.02808/) (2023)
- [Perceptual Quality Assessment for Digital Human Heads](https://mlsanity.com/paper/2209.09489/) (2022)
- [MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis](https://mlsanity.com/paper/2411.11235/) (2024)
- [JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generation](https://mlsanity.com/paper/2411.09209/) (2024)

## Built on by (most cited first, 4 of 4)

- [LongCat-Video-Avatar 1.5 Technical Report](https://mlsanity.com/paper/2605.26486/) (2026)
- [SocialDirector: Training-Free Social Interaction Control for Multi-Person Video Generation](https://mlsanity.com/paper/2605.10079/) (2026)
- [Audio-Visual Intelligence in Large Foundation Models](https://mlsanity.com/paper/2605.04045/) (2026)
- [HelloWorld: Enabling Socially Interactive Characters in Video World Models](https://mlsanity.com/paper/2608.05070/) (2026)

## Abstract (arXiv)

Speech-driven Talking Human (TH) generation, commonly known as "Talker," currently faces limitations in multi-subject driving capabilities. Extending this paradigm to "Multi-Talker," capable of animating multiple subjects simultaneously, introduces richer interactivity and stronger immersion in audiovisual communication. However, current Multi-Talkers still exhibit noticeable quality degradation caused by technical limitations, resulting in suboptimal user experiences. To address this challenge, we construct THQA-MT, the first large-scale Multi-Talker-generated Talking Human Quality Assessment dataset, consisting of 5,492 Multi-Talker-generated THs (MTHs) from 15 representative Multi-Talkers using 400 real portraits collected online. Through subjective experiments, we analyze perceptual discrepancies among different Multi-Talkers and identify 12 common types of distortion. Furthermore, we introduce EvalTalker, a novel TH quality assessment framework. This framework possesses the ability to perceive global quality, human characteristics, and identity consistency, while integrating Qwen-Sync to perceive multimodal synchrony. Experimental results demonstrate that EvalTalker achieves superior correlation with subjective scores, providing a robust foundation for future research on high-quality Multi-Talker generation and evaluation.