# HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation

arXiv: https://arxiv.org/abs/2403.09326 · HTML: https://mlsanity.com/paper/2403.09326/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 72 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022)
- [LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching](https://mlsanity.com/paper/2311.11284/) (2023)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D](https://mlsanity.com/paper/2311.16918/) (2023)
- [DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models](https://mlsanity.com/paper/2304.00916/) (2023)
- [Morphable Face Models - An Open Framework](https://mlsanity.com/paper/1709.08398/) (2017)
- [TADA! Text to Animatable Digital Avatars](https://mlsanity.com/paper/2308.10899/) (2023)
- [Neural Cages for Detail-Preserving 3D Deformations](https://mlsanity.com/paper/1912.06395/) (2019)
- [DreamHuman: Animatable 3D Avatars from Text](https://mlsanity.com/paper/2306.09329/) (2023)
- [Learning Formation of Physically-Based Face Attributes](https://mlsanity.com/paper/2004.03458/) (2020)
- [TextDeformer: Geometry Manipulation using Text Guidance](https://mlsanity.com/paper/2304.13348/) (2023)
- [HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation](https://mlsanity.com/paper/2310.01406/) (2023)
- [AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control](https://mlsanity.com/paper/2303.17606/) (2023)
- [DreamWaltz: Make a Scene with Complex 3D Animatable Avatars](https://mlsanity.com/paper/2305.12529/) (2023)
- [DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance](https://mlsanity.com/paper/2304.03117/) (2023)

## Built on by (most cited first, 11 of 11)

- [LLMs Meet Multimodal Generation and Editing: A Survey](https://mlsanity.com/paper/2405.19334/) (2024)
- [UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections](https://mlsanity.com/paper/2509.24817/) (2025)
- [Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondence](https://mlsanity.com/paper/2503.21766/) (2025)
- [StrandHead: Text to Hair-Disentangled 3D Head Avatars Using Human-Centric Priors](https://mlsanity.com/paper/2412.11586/) (2024)
- [T2Bs: Text-to-Character Blendshapes via Video Generation](https://mlsanity.com/paper/2509.10678/) (2025)
- [OmniFit: Multi-modal 3D Body Fitting via Scale-agnostic Dense Landmark Prediction](https://mlsanity.com/paper/2604.21575/) (2026)
- [Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculpting](https://mlsanity.com/paper/2502.20045/) (2025)
- [StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translation](https://mlsanity.com/paper/2508.11203/) (2025)
- [PoseAlign: Sculpting Pose-Consistent Meshes via Text-Guided Deformation](https://mlsanity.com/paper/2607.10560/) (2026)
- [RegHead: Non-Humanoid Head Blendshapes via Feed-Forward Registration](https://mlsanity.com/paper/2607.12206/) (2026)
- [RADmesh: Remesh-Aware Mesh Deformation](https://mlsanity.com/paper/2608.17182/) (2026)

## Abstract (arXiv)

Current text-to-avatar methods often rely on implicit representations (e.g., NeRF, SDF, and DMTet), leading to 3D content that artists cannot easily edit and animate in graphics software. This paper introduces a novel framework for generating stylized head avatars from text guidance, which leverages locally learnable mesh deformation and 2D diffusion priors to achieve high-quality digital assets for attribute-preserving manipulation. Given a template mesh, our method represents mesh deformation with per-face Jacobians and adaptively modulates local deformation using a learnable vector field. This vector field enables anisotropic scaling while preserving the rotation of vertices, which can better express identity and geometric details. We employ landmark- and contour-based regularization terms to balance the expressiveness and plausibility of generated avatars from multiple views without relying on any specific shape prior. Our framework can generate realistic shapes and textures that can be further edited via text, while supporting seamless editing using the preserved attributes from the template mesh, such as 3DMM parameters, blendshapes, and UV coordinates. Extensive experiments demonstrate that our framework can generate diverse and expressive head avatars with high-quality meshes that artists can easily manipulate in graphics software, facilitating downstream applications such as efficient asset creation and animation with preserved attributes.