# Facial Expression Retargeting from Human to Avatar Made Easy

arXiv: https://arxiv.org/abs/2008.05110 · HTML: https://mlsanity.com/paper/2008.05110/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2020.

## Built on by (most cited first, 13 of 13)

- [Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video](https://mlsanity.com/paper/2210.06108/) (2022)
- [FaceXHuBERT: Text-less Speech-driven E(X)pressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2303.05416/) (2023)
- [Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition](https://mlsanity.com/paper/2107.03708/) (2021)
- [4D Facial Expression Diffusion Model](https://mlsanity.com/paper/2303.16611/) (2023)
- [Everything's Talkin': Pareidolia Face Reenactment](https://mlsanity.com/paper/2104.03061/) (2021)
- [Versatile Face Animator: Driving Arbitrary 3D Facial Avatar in RGBD Space](https://mlsanity.com/paper/2308.06076/) (2023)
- [FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Model](https://mlsanity.com/paper/2409.13180/) (2024)
- [An Implicit Physical Face Model Driven by Expression and Style](https://mlsanity.com/paper/2401.15414/) (2024)
- [Deep Learning Based Facial Retargeting Using Local Patches](https://mlsanity.com/paper/2601.08429/) (2026)
- [Neuromuscular Control of the Face-Head-Neck Biomechanical Complex With Learning-Based Expression Transfer From Images and Videos](https://mlsanity.com/paper/2111.06517/) (2021)
- [MienCap: Realtime Performance-Based Facial Animation with Live Mood Dynamics](https://mlsanity.com/paper/2508.04687/) (2025)
- [Improving Facial Rig Semantics for Tracking and Retargeting](https://mlsanity.com/paper/2508.08429/) (2025)
- [Learning Domain Agnostic Latent Embeddings of 3D Faces for Zero-shot Animal Expression Transfer](https://mlsanity.com/paper/2601.06484/) (2026)

## Abstract (arXiv)

Facial expression retargeting from humans to virtual characters is a useful technique in computer graphics and animation. Traditional methods use markers or blendshapes to construct a mapping between the human and avatar faces. However, these approaches require a tedious 3D modeling process, and the performance relies on the modelers' experience. In this paper, we propose a brand-new solution to this cross-domain expression transfer problem via nonlinear expression embedding and expression domain translation. We first build low-dimensional latent spaces for the human and avatar facial expressions with variational autoencoder. Then we construct correspondences between the two latent spaces guided by geometric and perceptual constraints. Specifically, we design geometric correspondences to reflect geometric matching and utilize a triplet data structure to express users' perceptual preference of avatar expressions. A user-friendly method is proposed to automatically generate triplets for a system allowing users to easily and efficiently annotate the correspondences. Using both geometric and perceptual correspondences, we trained a network for expression domain translation from human to avatar. Extensive experimental results and user studies demonstrate that even nonprofessional users can apply our method to generate high-quality facial expression retargeting results with less time and effort.