# AU-Blendshape for Fine-grained Stylized 3D Facial Expression Manipulation

arXiv: https://arxiv.org/abs/2507.12001 · HTML: https://mlsanity.com/paper/2507.12001/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 39 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Capture, Learning, and Synthesis of 3D Speaking Styles](https://mlsanity.com/paper/1905.03079/) (2019)
- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [Learning Formation of Physically-Based Face Attributes](https://mlsanity.com/paper/2004.03458/) (2020)
- [Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesis](https://mlsanity.com/paper/2211.14506/) (2022)
- [FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion](https://mlsanity.com/paper/2309.11306/) (2023)
- [Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model](https://mlsanity.com/paper/2210.12315/) (2022)
- [FEAFA: A Well-Annotated Dataset for Facial Expression Analysis and 3D Facial Animation](https://mlsanity.com/paper/1904.01509/) (2019)
- [FEAFA+: An Extended Well-Annotated Dataset for Facial Expression Analysis and 3D Facial Animation](https://mlsanity.com/paper/2111.02751/) (2021)
- [3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning](https://mlsanity.com/paper/2009.14798/) (2020)

## Built on by (most cited first, 3 of 3)

- [RAM: Recover Any 3D Human Motion in-the-Wild](https://mlsanity.com/paper/2603.19929/) (2026)
- [CNSocialDepress: A Chinese Social Media Dataset for Depression Risk Detection and Structured Analysis](https://mlsanity.com/paper/2510.11233/) (2025)
- [TopoRig: Topology-Agnostic Facial Rigging via Multi-Source Supervision](https://mlsanity.com/paper/2609.15746/) (2026)

## Abstract (arXiv)

While 3D facial animation has made impressive progress, challenges still exist in realizing fine-grained stylized 3D facial expression manipulation due to the lack of appropriate datasets. In this paper, we introduce the AUBlendSet, a 3D facial dataset based on AU-Blendshape representation for fine-grained facial expression manipulation across identities. AUBlendSet is a blendshape data collection based on 32 standard facial action units (AUs) across 500 identities, along with an additional set of facial postures annotated with detailed AUs. Based on AUBlendSet, we propose AUBlendNet to learn AU-Blendshape basis vectors for different character styles. AUBlendNet predicts, in parallel, the AU-Blendshape basis vectors of the corresponding style for a given identity mesh, thereby achieving stylized 3D emotional facial manipulation. We comprehensively validate the effectiveness of AUBlendSet and AUBlendNet through tasks such as stylized facial expression manipulation, speech-driven emotional facial animation, and emotion recognition data augmentation. Through a series of qualitative and quantitative experiments, we demonstrate the potential and importance of AUBlendSet and AUBlendNet in 3D facial animation tasks. To the best of our knowledge, AUBlendSet is the first dataset, and AUBlendNet is the first network for continuous 3D facial expression manipulation for any identity through facial AUs. Our source code is available at https://github.com/wslh852/AUBlendNet.git.