# SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting

arXiv: https://arxiv.org/abs/2403.05087 · HTML: https://mlsanity.com/paper/2403.05087/

## Key facts

- Cited by 172 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 54 references that are in the map.
- First posted on arXiv 8 Mar 2024.

## Builds on (most cited first, 30 of 36)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis](https://mlsanity.com/paper/2308.09713/) (2023)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [Mixture of Volumetric Primitives for Efficient Neural Rendering](https://mlsanity.com/paper/2103.01954/) (2021)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [PointAvatar: Deformable Point-based Head Avatars from Videos](https://mlsanity.com/paper/2212.08377/) (2022)
- [Generating Holistic 3D Human Motion from Speech](https://mlsanity.com/paper/2212.04420/) (2022)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion](https://mlsanity.com/paper/2110.13746/) (2021)
- [InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds](https://mlsanity.com/paper/2212.10550/) (2022)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition](https://mlsanity.com/paper/2302.11566/) (2023)
- [Pixel Codec Avatars](https://mlsanity.com/paper/2104.04638/) (2021)
- [Detailed Human Avatars from Monocular Video](https://mlsanity.com/paper/1808.01338/) (2018)
- [Animatable Neural Radiance Fields from Monocular RGB Videos](https://mlsanity.com/paper/2106.13629/) (2021)
- [MonoHuman: Animatable Human Neural Field from Monocular Video](https://mlsanity.com/paper/2304.02001/) (2023)
- [AvatarReX: Real-time Expressive Full-body Avatars](https://mlsanity.com/paper/2305.04789/) (2023)
- [Capturing and Animation of Body and Clothing from Monocular Video](https://mlsanity.com/paper/2210.01868/) (2022)
- [SMPLpix: Neural Avatars from 3D Human Models](https://mlsanity.com/paper/2008.06872/) (2020)
- [Modeling Clothing as a Separate Layer for an Animatable Human Avatar](https://mlsanity.com/paper/2106.14879/) (2021)
- [Fast-SNARF: A Fast Deformer for Articulated Neural Fields](https://mlsanity.com/paper/2211.15601/) (2022)
- [Learning Locally Editable Virtual Humans](https://mlsanity.com/paper/2305.00121/) (2023)

## Built on by (most cited first, 30 of 172)

- [MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds](https://mlsanity.com/paper/2405.17421/) (2024)
- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [Wild-GS: Real-Time Novel View Synthesis from Unconstrained Photo Collections](https://mlsanity.com/paper/2406.10373/) (2024)
- [HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors](https://mlsanity.com/paper/2406.12459/) (2024)
- [CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models](https://mlsanity.com/paper/2412.12093/) (2024)
- [Generative Gaussian Splatting for Unbounded 3D City Generation](https://mlsanity.com/paper/2406.06526/) (2024)
- [HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors](https://mlsanity.com/paper/2408.06019/) (2024)
- [Textured Gaussians for Enhanced 3D Scene Appearance Modeling](https://mlsanity.com/paper/2411.18625/) (2024)
- [GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion](https://mlsanity.com/paper/2412.10209/) (2024)
- [4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models](https://mlsanity.com/paper/2503.10437/) (2025)
- [FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video](https://mlsanity.com/paper/2411.15604/) (2024)
- [Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior](https://mlsanity.com/paper/2503.01610/) (2025)
- [RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars](https://mlsanity.com/paper/2503.12886/) (2025)
- [LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priors](https://mlsanity.com/paper/2409.03456/) (2024)
- [Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video](https://mlsanity.com/paper/2409.08189/) (2024)
- [HRAvatar: High-Quality and Relightable Gaussian Head Avatar](https://mlsanity.com/paper/2503.08224/) (2025)
- [GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian Splatting](https://mlsanity.com/paper/2404.14037/) (2024)
- [DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characters](https://mlsanity.com/paper/2411.17423/) (2024)
- [HiMoR: Monocular Deformable Gaussian Reconstruction with Hierarchical Motion Representation](https://mlsanity.com/paper/2504.06210/) (2025)
- [GUAVA: Generalizable Upper Body 3D Gaussian Avatar](https://mlsanity.com/paper/2505.03351/) (2025)
- [Expressive Gaussian Human Avatars from Monocular RGB Video](https://mlsanity.com/paper/2407.03204/) (2024)
- [TextToon: Real-Time Text Toonify Head Avatar from Single Video](https://mlsanity.com/paper/2410.07160/) (2024)
- [GaussianDreamerPro: Text to Manipulable 3D Gaussians with Highly Enhanced Quality](https://mlsanity.com/paper/2406.18462/) (2024)
- [3D-GSW: 3D Gaussian Splatting for Robust Watermarking](https://mlsanity.com/paper/2409.13222/) (2024)
- [GuardSplat: Efficient and Robust Watermarking for 3D Gaussian Splatting](https://mlsanity.com/paper/2411.19895/) (2024)
- [Disco4D: Disentangled 4D Human Generation and Animation from a Single Image](https://mlsanity.com/paper/2409.17280/) (2024)
- [MVP4D: Multi-View Portrait Video Diffusion for Animatable 4D Avatars](https://mlsanity.com/paper/2510.12785/) (2025)
- [Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance](https://mlsanity.com/paper/2501.05379/) (2025)
- [SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatars](https://mlsanity.com/paper/2410.11682/) (2024)
- [PERSE: Personalized 3D Generative Avatars from A Single Portrait](https://mlsanity.com/paper/2412.21206/) (2024)

## Abstract (arXiv)

We present SplattingAvatar, a hybrid 3D representation of photorealistic human avatars with Gaussian Splatting embedded on a triangle mesh, which renders over 300 FPS on a modern GPU and 30 FPS on a mobile device. We disentangle the motion and appearance of a virtual human with explicit mesh geometry and implicit appearance modeling with Gaussian Splatting. The Gaussians are defined by barycentric coordinates and displacement on a triangle mesh as Phong surfaces. We extend lifted optimization to simultaneously optimize the parameters of the Gaussians while walking on the triangle mesh. SplattingAvatar is a hybrid representation of virtual humans where the mesh represents low-frequency motion and surface deformation, while the Gaussians take over the high-frequency geometry and detailed appearance. Unlike existing deformation methods that rely on an MLP-based linear blend skinning (LBS) field for motion, we control the rotation and translation of the Gaussians directly by mesh, which empowers its compatibility with various animation techniques, e.g., skeletal animation, blend shapes, and mesh editing. Trainable from monocular videos for both full-body and head avatars, SplattingAvatar shows state-of-the-art rendering quality across multiple datasets.