# HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2402.06149 · HTML: https://mlsanity.com/paper/2402.06149/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 89 references that are in the map.
- First posted on arXiv 9 Feb 2024.

## Builds on (most cited first, 30 of 71)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)

## Built on by (most cited first, 30 of 35)

- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [Generalizable Human Gaussians for Sparse View Synthesis](https://mlsanity.com/paper/2407.12777/) (2024)
- [UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modeling](https://mlsanity.com/paper/2403.11589/) (2024)
- [Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospects](https://mlsanity.com/paper/2412.06257/) (2024)
- [VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation](https://mlsanity.com/paper/2407.09822/) (2024)
- [6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering](https://mlsanity.com/paper/2410.04974/) (2024)
- [A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generation](https://mlsanity.com/paper/2406.04253/) (2024)
- [GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting](https://mlsanity.com/paper/2410.05259/) (2024)
- [Few-shot Novel View Synthesis using Depth Aware 3D Gaussian Splatting](https://mlsanity.com/paper/2410.11080/) (2024)
- [GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splats](https://mlsanity.com/paper/2406.02968/) (2024)
- [Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video Diffusion](https://mlsanity.com/paper/2503.15851/) (2025)
- [X-Oscar: A Progressive Framework for High-quality Text-guided 3D Animatable Avatar Generation](https://mlsanity.com/paper/2405.00954/) (2024)
- [A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing](https://mlsanity.com/paper/2406.10553/) (2024)
- [FaceCraft4D: Animated 3D Facial Avatar Generation from a Single Image](https://mlsanity.com/paper/2504.15179/) (2025)
- [StrandHead: Text to Hair-Disentangled 3D Head Avatars Using Human-Centric Priors](https://mlsanity.com/paper/2412.11586/) (2024)
- [SpinMeRound: Consistent Multi-View Identity Generation Using Diffusion Models](https://mlsanity.com/paper/2504.10716/) (2025)
- [Creating Your Editable 3D Photorealistic Avatar with Tetrahedron-constrained Gaussian Splatting](https://mlsanity.com/paper/2504.20403/) (2025)
- [PlantDreamer: Achieving Realistic 3D Plant Models with Diffusion-Guided Gaussian Splatting](https://mlsanity.com/paper/2505.15528/) (2025)
- [Bringing Your Portrait to 3D Presence](https://mlsanity.com/paper/2511.22553/) (2025)
- [AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatars](https://mlsanity.com/paper/2512.06438/) (2025)
- [T2Bs: Text-to-Character Blendshapes via Video Generation](https://mlsanity.com/paper/2509.10678/) (2025)
- [BalanceGS: Algorithm-System Co-design for Efficient 3D Gaussian Splatting Training on GPU](https://mlsanity.com/paper/2510.14564/) (2025)
- [ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splatting](https://mlsanity.com/paper/2601.01847/) (2026)
- [Face Anything: 4D Face Reconstruction from Any Image Sequence](https://mlsanity.com/paper/2604.19702/) (2026)
- [Refining Gaussian Splatting: A Volumetric Densification Approach](https://mlsanity.com/paper/2508.05187/) (2025)
- [StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translation](https://mlsanity.com/paper/2508.11203/) (2025)
- [ExposeAnyone: Personalized Audio-to-Expression Diffusion Models Are Robust Zero-Shot Face Forgery Detectors](https://mlsanity.com/paper/2601.02359/) (2026)
- [From Blurry to Believable: Enhancing Low-quality Talking Heads with 3D Generative Priors](https://mlsanity.com/paper/2602.06122/) (2026)
- [EmoteGPT: 3D Human Facial Expressions from Natural Language Descriptions](https://mlsanity.com/paper/2607.02674/) (2026)
- [GDGS: Gradient Domain Gaussian Splatting for Sparse Representation of Radiance Fields](https://mlsanity.com/paper/2405.05446/) (2024)

## Abstract (arXiv)

Creating digital avatars from textual prompts has long been a desirable yet challenging task. Despite the promising results achieved with 2D diffusion priors, current methods struggle to create high-quality and consistent animated avatars efficiently. Previous animatable head models like FLAME have difficulty in accurately representing detailed texture and geometry. Additionally, high-quality 3D static representations face challenges in semantically driving with dynamic priors. In this paper, we introduce HeadStudio, a novel framework that utilizes 3D Gaussian splatting to generate realistic and animatable avatars from text prompts. Firstly, we associate 3D Gaussians with animatable head prior model, facilitating semantic animation on high-quality 3D representations. To ensure consistent animation, we further enhance the optimization from initialization, distillation, and regularization to jointly learn the shape, texture, and animation. Extensive experiments demonstrate the efficacy of HeadStudio in generating animatable avatars from textual prompts, exhibiting appealing appearances. The avatars are capable of rendering high-quality real-time (≥ 40 fps) novel views at a resolution of 1024. Moreover, These avatars can be smoothly driven by real-world speech and video. We hope that HeadStudio can enhance digital avatar creation and gain popularity in the community. Code is at: https://github.com/ZhenglinZhou/HeadStudio.