mlsanityData to 13 Oct 2026

AvatarReX: Real-time Expressive Full-body Avatars

arXiv 2305.04789 ↗First posted 8 May 2023cs.CV cs.GRcited by 83

Not on the map. Why?

Abstract (arXiv)

We present AvatarReX, a new method for learning NeRF-based full-body avatars from video data. The learnt avatar not only provides expressive control of the body, hands and the face together, but also supports real-time animation and rendering. To this end, we propose a compositional avatar representation, where the body, hands and the face are separately modeled in a way that the structural prior from parametric mesh templates is properly utilized without compromising representation flexibility. Furthermore, we disentangle the geometry and appearance for each part. With these technical designs, we propose a dedicated deferred rendering pipeline, which can be executed in real-time framerate to synthesize high-quality free-view images. The disentanglement of geometry and appearance also allows us to design a two-pass training strategy that combines volume rendering and surface rendering for network training. In this way, patch-level supervision can be applied to force the network to learn sharp appearance details on the basis of geometry estimation. Overall, our method enables automatic construction of expressive full-body avatars with real-time rendering capability, and can generate photo-realistic images with dynamic details for novel body motions and facial expressions.

Key facts. Cited by 83 papers in the map (data through 13 Oct 2026). Builds on 81 of its 109 references that are in the map. First posted on arXiv 8 May 2023.

Builds on

81 of 109 references.

References · 81

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,712
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,686
  4. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,183
  5. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,610
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,603
  7. 2022TensoRF: Tensorial Radiance Fieldscited by 1,277
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  9. 2021Volume Rendering of Neural Implicit Surfacescited by 992
  10. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 962
  11. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  12. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
38 more
  1. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  2. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 495
  3. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  4. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 476
  5. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 466
  6. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  7. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  8. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 344
  9. 2021AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesiscited by 339
  10. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 311
  11. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  12. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  13. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  14. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  15. 2018Video Based Reconstruction of 3D People Modelscited by 237
  16. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  17. 2021I M Avatar: Implicit Morphable Head Avatars from Videoscited by 230
  18. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  19. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  20. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  21. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  22. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  23. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  24. 2020Real-Time High-Resolution Background Mattingcited by 172
  25. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  26. 2022Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-timecited by 165
  27. 2018Deep Appearance Models for Face Renderingcited by 164
  28. 2020DeepCap: Monocular Human Performance Capture Using Weak Supervisioncited by 162
  29. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  30. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  31. 2021Real-time Deep Dynamic Characterscited by 133
  32. 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
  33. 2021Pixel Codec Avatarscited by 126
  34. 2022Structured Local Radiance Fields for Human Avatar Modelingcited by 116
  35. 2018Detailed Human Avatars from Monocular Videocited by 109
  36. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  37. 2021LEAP: Learning Articulated Occupancy of Peoplecited by 107
  38. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 23; 2024: 21; 2025: 21; 2026: 18.

83 papers.

Sort:

By area:

  1. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 308
  2. 2023GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussianscited by 190
  3. 2024SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splattingcited by 172
  4. 2023TADA! Text to Animatable Digital Avatarscited by 110
  5. 2023Human Gaussian Splatting: Real-time Rendering of Animatable Avatarscited by 105
  6. 2023ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Renderingcited by 104
  7. 2023HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splattingcited by 69
  8. 2023GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learningcited by 63
  9. 2023AvatarVerse: High-quality & Stable 3D Avatar Creation from Text and Posecited by 60
  10. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  11. 2024MeshAvatar: Learning High-quality Triangular Human Avatars from Multi-view Videoscited by 34
  12. 2023Control4D: Efficient 4D Portrait Editing with Textcited by 32
38 more
  1. 2023Human101: Training 100+FPS Human Gaussians in 100s from 1 Viewcited by 32
  2. 2025Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Priorcited by 31
  3. 2024HAHA: Highly Articulated Gaussian Human Avatars with Textured Mesh Priorcited by 30
  4. 2023Animatable and Relightable Gaussians for High-fidelity Human Avatar Modelingcited by 27
  5. 2023VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streamscited by 22
  6. 2024LayGA: Layered Gaussian Avatars for Animatable Clothing Transfercited by 22
  7. 2024Expressive Gaussian Human Avatars from Monocular RGB Videocited by 21
  8. 2023CaPhy: Capturing Physical Properties for Animatable Human Avatarscited by 20
  9. 2024Gaussian Shadow Casting for Neural Characterscited by 14
  10. 2023Relightable and Animatable Neural Avatars from Videoscited by 13
  11. 2025HumanDreamer-X: Photorealistic Single-image Human Avatars Reconstruction via Gaussian Restorationcited by 13
  12. 2025PERSONA: Personalized Whole-Body 3D Avatar with Pose-Driven Deformations from a Single Imagecited by 13
  13. 2023HumanNeRF-SE: A Simple yet Effective Approach to Animate HumanNeRF with Diverse Posescited by 12
  14. 2024Neural Radiance Field-based Visual Rendering: A Comprehensive Reviewcited by 12
  15. 2025Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPscited by 12
  16. 2024Within the Dynamic Context: Inertia-aware 3D Human Modeling with Pose Sequencecited by 11
  17. 2025TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented Reality via 3D Gaussian Splattingcited by 11
  18. 2024DEGAS: Detailed Expressions on Full-Body Gaussian Avatarscited by 10
  19. 2024SqueezeMe: Mobile-Ready Distillation of Gaussian Full-Body Avatarscited by 10
  20. 2023High-Quality Facial Geometry and Appearance Capture at Homecited by 9
  21. 2024A Survey on 3D Egocentric Human Pose Estimationcited by 8
  22. 2025RePerformer: Immersive Human-centric Volumetric Videos from Playback to Photoreal Reperformancecited by 8
  23. 2024EgoAvatar: Egocentric View-Driven and Photorealistic Full-body Avatarscited by 7
  24. 2025EVA: Expressive Virtual Avatars from Multi-view Videoscited by 7
  25. 2025Interspatial Attention for Efficient 4D Human Video Generationcited by 7
  26. 2023Leveraging Intrinsic Properties for Non-Rigid Garment Alignmentcited by 6
  27. 2023Sounding Bodies: Modeling 3D Spatial Sound of Humans Using Body Pose and Audiocited by 6
  28. 2024Interactive Rendering of Relightable and Animatable Gaussian Avatarscited by 6
  29. 2025BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Videocited by 5
  30. 2026ReconPhys: Reconstruct Appearance and Physical Attributes from Single Videocited by 5
  31. 2023SpeechAct: Towards Generating Whole-body Motion from Speechcited by 4
  32. 2024TexVocab: Texture Vocabulary-conditioned Human Avatarscited by 4
  33. 2024Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modelingcited by 4
  34. 2024TalkinNeRF: Animatable Neural Fields for Full-Body Talking Humanscited by 4
  35. 2025MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesiscited by 4
  36. 2025AHA! Animating Human Avatars in Diverse Scenes with Gaussian Splattingcited by 4
  37. 2023Pose Modulated Avatars from Videocited by 3
  38. 2023Animatable Virtual Humans: Learning pose-dependent human representations in UV space for interactive performance synthesiscited by 3

50 most cited of 83 All 83 (JSON)