# Expressive Whole-Body 3D Gaussian Avatar

arXiv: https://arxiv.org/abs/2407.21686 · HTML: https://mlsanity.com/paper/2407.21686/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 54 references that are in the map.
- First posted on arXiv 31 Jul 2024.

## Builds on (most cited first, 30 of 47)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering](https://mlsanity.com/paper/2311.12775/) (2023)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model](https://mlsanity.com/paper/2311.16498/) (2023)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Video Based Reconstruction of 3D People Models](https://mlsanity.com/paper/1803.04758/) (2018)
- [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021)
- [3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting](https://mlsanity.com/paper/2312.09228/) (2023)
- [GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians](https://mlsanity.com/paper/2312.02134/) (2023)
- [FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration](https://mlsanity.com/paper/2108.06428/) (2021)
- [Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering](https://mlsanity.com/paper/2109.07448/) (2021)
- [Monocular Expressive Body Regression through Body-Driven Attention](https://mlsanity.com/paper/2008.09062/) (2020)
- [SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2309.17448/) (2023)

## Built on by (most cited first, 30 of 89)

- [Visual Imitation Enables Contextual Humanoid Control](https://mlsanity.com/paper/2505.03729/) (2025)
- [LHM: Large Animatable Human Reconstruction Model from a Single Image in Seconds](https://mlsanity.com/paper/2503.10625/) (2025)
- [Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior](https://mlsanity.com/paper/2503.01610/) (2025)
- [GUAVA: Generalizable Upper Body 3D Gaussian Avatar](https://mlsanity.com/paper/2505.03351/) (2025)
- [Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance](https://mlsanity.com/paper/2501.05379/) (2025)
- [RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videos](https://mlsanity.com/paper/2503.16822/) (2025)
- [PERSONA: Personalized Whole-Body 3D Avatar with Pose-Driven Deformations from a Single Image](https://mlsanity.com/paper/2508.09973/) (2025)
- [DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D Diffusion](https://mlsanity.com/paper/2409.17145/) (2024)
- [RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2503.14198/) (2025)
- [Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs](https://mlsanity.com/paper/2504.12909/) (2025)
- [BIGS: Bimanual Category-agnostic Interaction Reconstruction from Monocular Videos via 3D Gaussian Splatting](https://mlsanity.com/paper/2504.09097/) (2025)
- [HumanRAM: Feed-forward Human Reconstruction and Animation Model using Transformers](https://mlsanity.com/paper/2506.03118/) (2025)
- [MoGA: 3D Generative Avatar Prior for Monocular Gaussian Avatar Reconstruction](https://mlsanity.com/paper/2507.23597/) (2025)
- [Disentangled Clothed Avatar Generation with Layered Representation](https://mlsanity.com/paper/2501.04631/) (2025)
- [EVA: Expressive Virtual Avatars from Multi-view Videos](https://mlsanity.com/paper/2505.15385/) (2025)
- [WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction](https://mlsanity.com/paper/2502.01045/) (2025)
- [SignSplat: Rendering Sign Language via Gaussian Splatting](https://mlsanity.com/paper/2505.02108/) (2025)
- [LHM++: An Efficient Large Human Reconstruction Model for Pose-free Images to 3D](https://mlsanity.com/paper/2506.13766/) (2025)
- [Learning to Ball: Composing Policies for Long-Horizon Basketball Moves](https://mlsanity.com/paper/2509.22442/) (2025)
- [GASPACHO: Gaussian Splatting for Controllable Humans and Objects](https://mlsanity.com/paper/2503.09342/) (2025)
- [Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Field](https://mlsanity.com/paper/2505.10049/) (2025)
- [SAT: Supervisor Regularization and Animation Augmentation for Two-process Monocular Texture 3D Human Reconstruction](https://mlsanity.com/paper/2508.19688/) (2025)
- [ContraGS: Codebook-Condensed and Trainable Gaussian Splatting for Fast, Memory-Efficient Reconstruction](https://mlsanity.com/paper/2509.03775/) (2025)
- [AHA! Animating Human Avatars in Diverse Scenes with Gaussian Splatting](https://mlsanity.com/paper/2511.09827/) (2025)
- [R3-Avatar: Record and Retrieve Temporal Codebook for Reconstructing Photorealistic Human Avatars](https://mlsanity.com/paper/2503.12751/) (2025)
- [Parametric Gaussian Human Model: Generalizable Prior for Efficient and Realistic Human Avatar Modeling](https://mlsanity.com/paper/2506.06645/) (2025)
- [MonoCloth: Reconstruction and Animation of Cloth-Decoupled Human Avatars from Monocular Videos](https://mlsanity.com/paper/2508.04505/) (2025)
- [PA-HOI: A Physics-Aware Human and Object Interaction Dataset](https://mlsanity.com/paper/2508.06205/) (2025)
- [SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit 3D Meshes](https://mlsanity.com/paper/2509.21859/) (2025)
- [Bringing Your Portrait to 3D Presence](https://mlsanity.com/paper/2511.22553/) (2025)

## Abstract (arXiv)

Facial expression and hand motions are necessary to express our emotions and interact with the world. Nevertheless, most of the 3D human avatars modeled from a casually captured video only support body motions without facial expressions and hand motions.In this work, we present ExAvatar, an expressive whole-body 3D human avatar learned from a short monocular video. We design ExAvatar as a combination of the whole-body parametric mesh model (SMPL-X) and 3D Gaussian Splatting (3DGS). The main challenges are 1) a limited diversity of facial expressions and poses in the video and 2) the absence of 3D observations, such as 3D scans and RGBD images. The limited diversity in the video makes animations with novel facial expressions and poses non-trivial. In addition, the absence of 3D observations could cause significant ambiguity in human parts that are not observed in the video, which can result in noticeable artifacts under novel motions. To address them, we introduce our hybrid representation of the mesh and 3D Gaussians. Our hybrid representation treats each 3D Gaussian as a vertex on the surface with pre-defined connectivity information (i.e., triangle faces) between them following the mesh topology of SMPL-X. It makes our ExAvatar animatable with novel facial expressions by driven by the facial expression space of SMPL-X. In addition, by using connectivity-based regularizers, we significantly reduce artifacts in novel facial expressions and poses.