# A multidimensional measurement of photorealistic avatar quality of experience

arXiv: https://arxiv.org/abs/2411.09066 · HTML: https://mlsanity.com/paper/2411.09066/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 82 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 17 of 17)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control](https://mlsanity.com/paper/2407.03168/) (2024)
- [VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time](https://mlsanity.com/paper/2404.10667/) (2024)
- [SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting](https://mlsanity.com/paper/2403.05087/) (2024)
- [DisCo: Disentangled Control for Realistic Human Dance Generation](https://mlsanity.com/paper/2307.00040/) (2023)
- [Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates](https://mlsanity.com/paper/2108.08020/) (2021)
- [DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars](https://mlsanity.com/paper/2311.18635/) (2023)
- [MegActor: Harness the Power of Raw Video for Vivid Portrait Animation](https://mlsanity.com/paper/2405.20851/) (2024)
- [An Open source Implementation of ITU-T Recommendation P.808 with Validation](https://mlsanity.com/paper/2005.08138/) (2020)
- [Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Framework](https://mlsanity.com/paper/2403.16510/) (2024)
- [ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applications](https://mlsanity.com/paper/2203.16032/) (2022)
- [UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Textures](https://mlsanity.com/paper/2401.11078/) (2024)
- [Disentangling Foreground and Background Motion for Enhanced Realism in Human Video Generation](https://mlsanity.com/paper/2405.16393/) (2024)
- [Full Reference Video Quality Assessment for Machine Learning-Based Video Codecs](https://mlsanity.com/paper/2309.00769/) (2023)
- [Subjective and Objective Quality Assessment of Rendered Human Avatar Videos in Virtual Reality](https://mlsanity.com/paper/2408.07041/) (2024)
- [Application of Just-Noticeable Difference in Quality as Environment Suitability Test for Crowdsourcing Speech Quality Assessment Task](https://mlsanity.com/paper/2004.05502/) (2020)
- [Real-and-Present: Investigating the Use of Life-Size 2D Video Avatars in HMD-Based AR Teleconferencing](https://mlsanity.com/paper/2401.02171/) (2024)

## Abstract (arXiv)

Photorealistic avatars are human avatars that look, move, and talk like real people. The performance of photorealistic avatars has significantly improved recently based on objective metrics such as PSNR, SSIM, LPIPS, FID, and FVD. However, recent photorealistic avatar publications do not provide subjective tests of the avatars to measure human usability factors. We provide an open source test framework to subjectively measure photorealistic avatar performance in ten dimensions: realism, trust, comfortableness using, comfortableness interacting with, appropriateness for work, creepiness, formality, affinity, resemblance to the person, and emotion accuracy. Using telecommunication scenarios, we show that the correlation of nine of these subjective metrics with PSNR, SSIM, LPIPS, FID, and FVD is weak, and moderate for emotion accuracy. The crowdsourced subjective test framework is highly reproducible and accurate when compared to a panel of experts. We analyze a wide range of avatars from photorealistic to cartoon-like and show that some photorealistic avatars are approaching real video performance based on these dimensions. We also find that for avatars above a certain level of realism, eight of these measured dimensions are strongly correlated. This means that avatars that are not as realistic as real video will have lower trust, comfortableness using, comfortableness interacting with, appropriateness for work, formality, and affinity, and higher creepiness compared to real video. In addition, because there is a strong linear relationship between avatar affinity and realism, there is no uncanny valley effect for photorealistic avatars in the telecommunication scenario. We suggest several extensions of this test framework for future work and discuss design implications for telecommunication systems. The test framework is available at https://github.com/microsoft/P.910.