mlsanityData to 14 Oct 2026

PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observations

arXiv 2404.04421 ↗First posted 5 Apr 2024cs.GR cs.CVcited by 51

Not on the map. Why?

Abstract (arXiv)

Modeling and rendering photorealistic avatars is of crucial importance in many applications. Existing methods that build a 3D avatar from visual observations, however, struggle to reconstruct clothed humans. We introduce PhysAvatar, a novel framework that combines inverse rendering with inverse physics to automatically estimate the shape and appearance of a human from multi-view video data along with the physical parameters of the fabric of their clothes. For this purpose, we adopt a mesh-aligned 4D Gaussian technique for spatio-temporal mesh tracking as well as a physically based inverse renderer to estimate the intrinsic material properties. PhysAvatar integrates a physics simulator to estimate the physical parameters of the garments using gradient-based optimization in a principled manner. These novel capabilities enable PhysAvatar to create high-quality novel-view renderings of avatars dressed in loose-fitting clothes under motions and lighting conditions not seen in the training data. This marks a significant advancement towards modeling photorealistic digital humans using physically based inverse rendering with physics in the loop. Our project website is at: https://qingqing-zhao.github.io/PhysAvatar

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Builds on 86 of its 131 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

86 of 131 references.

References · 86

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  5. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  9. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  10. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  11. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  12. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
38 more
  1. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 737
  2. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  3. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  4. 2017Neural 3D Mesh Renderercited by 437
  5. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  6. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 405
  7. 2021Advances in Neural Renderingcited by 365
  8. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  9. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 311
  10. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  11. 2020State of the Art on Neural Renderingcited by 278
  12. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
  13. 2019Differentiable Surface Splatting for Point-based Geometry Processingcited by 239
  14. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  15. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  16. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 217
  17. 2021Neural Articulated Radiance Fieldcited by 196
  18. 2023GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussianscited by 191
  19. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  20. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  21. 2023HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motioncited by 161
  22. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  23. 2022ARAH: Animatable Volume Rendering of Articulated Human SDFscited by 139
  24. 2021Real-time Deep Dynamic Characterscited by 133
  25. 2023Drivable 3D Gaussian Avatarscited by 120
  26. 2019CLOTH3D: Clothed 3D Humanscited by 116
  27. 2022Structured Local Radiance Fields for Human Avatar Modelingcited by 116
  28. 2023ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Renderingcited by 104
  29. 2021Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothingcited by 102
  30. 2022Generative Neural Articulated Radiance Fieldscited by 101
  31. 2021The Power of Points for Modeling Humans in Clothingcited by 90
  32. 2019Learning-Based Animation of Clothing for Virtual Try-Oncited by 89
  33. 2022SNUG: Self-Supervised Neural Dynamic Garmentscited by 83
  34. 2022Capturing and Animation of Body and Clothing from Monocular Videocited by 83
  35. 2021Driving-Signal Aware Full-Body Avatarscited by 81
  36. 2022HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamicscited by 78
  37. 2021SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elementscited by 72
  38. 2022gDNA: Towards Generative Detailed Neural Avatarscited by 72

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 22; 2026: 18.

51 papers.

Sort:

By area:

  1. 2025Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Priorcited by 31
  2. 2024Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Videocited by 25
  3. 2025Relightable Full-Body Gaussian Codec Avatarscited by 22
  4. 2025GauSTAR: Gaussian Surface Tracking and Reconstructioncited by 13
  5. 2025Articulated Kinematics Distillation from Video Diffusion Modelscited by 13
  6. 2024Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Videocited by 12
  7. 2024DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D Diffusioncited by 12
  8. 2024Multimodal Latent Diffusion Model for Complex Sewing Pattern Generationcited by 12
  9. 2025TaoAvatar: Real-Time Lifelike Full-Body Talking Avatars for Augmented Reality via 3D Gaussian Splattingcited by 11
  10. 2024ToMiE: Towards Explicit Exoskeleton for the Reconstruction of Complicated 3D Human Avatarscited by 10
  11. 2024Sequential Gaussian Avatars with Hierarchical Motion Contextcited by 9
  12. 2024PICA: Physics-Integrated Clothed Avatarcited by 8
38 more
  1. 2025MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamicscited by 8
  2. 2025RMAvatar: Photorealistic Human Avatar Reconstruction from Monocular Video Based on Rectified Mesh-embedded Gaussianscited by 7
  3. 2025GroomLight: Hybrid Inverse Rendering for Relightable Human Hair Appearance Modelingcited by 7
  4. 2025HumanOLAT: A Large-Scale Dataset for Full-Body Human Relighting and Novel-View Synthesiscited by 7
  5. 2026MOSIV: Multi-Object System Identification from Videoscited by 6
  6. 2025BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Videocited by 5
  7. 2024PokeFlex: A Real-World Dataset of Volumetric Deformable Objects for Roboticscited by 4
  8. 2024GGAvatar: Reconstructing Garment-Separated 3D Gaussian Splatting Avatars from Monocular Videocited by 4
  9. 2025ProJo4D: Progressive Joint Optimization for Sparse-View Inverse Physics Estimationcited by 4
  10. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  11. 2025Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fieldscited by 3
  12. 2025UMA: Ultra-detailed Human Avatars via Multi-level Surface Alignmentcited by 3
  13. 2025Parametric Gaussian Human Model: Generalizable Prior for Efficient and Realistic Human Avatar Modelingcited by 3
  14. 2025MonoCloth: Reconstruction and Animation of Cloth-Decoupled Human Avatars from Monocular Videoscited by 3
  15. 2026Image2Garment: Simulation-ready Garment Generation from a Single Imagecited by 3
  16. 2026NeuROK: Generative 4D Neural Object Kinematicscited by 3
  17. 2025SVAD: From Single Image to 3D Avatar via Synthetic Data Generation with Video Diffusion and Data Augmentationcited by 2
  18. 2025SpringTime: Learning Simulatable Models of Cloth with Spatially-varying Constitutive Propertiescited by 2
  19. 2025NGD: Neural Gradient Based Deformation for Monocular Garment Reconstructioncited by 2
  20. 2025Human Geometry Distribution for 3D Animation Generationcited by 2
  21. 2026Gaussian Wardrobe: Compositional 3D Gaussian Avatars for Free-Form Virtual Try-Oncited by 2
  22. 2026DAMA: Disentangled Body-Anchored Gaussians for Controllable Multi-Layered Avatarscited by 2
  23. 2024SOAR: Self-Occluded Avatar Recovery from a Single Video In the Wildcited by 1
  24. 2026Mon3tr: Monocular 3D Telepresence with Pre-built Gaussian Avatars as Amortizationcited by 1
  25. 2026ReWeaver: Towards Simulation-Ready and Topology-Accurate Garment Reconstructioncited by 1
  26. 2026Ani3DHuman: Photorealistic 3D Human Animation with Self-guided Stochastic Samplingcited by 1
  27. 2026InverseDraping: Recovering Sewing Patterns from 3D Garment Surfaces via BoxMesh Bridgingcited by 1
  28. 2026PIAvatar: Physically Interactive Avatars via Deformation Gradient Decouplingcited by 1
  29. 2024AIpparel: A Multimodal Foundation Model for Digital Garmentscited by 0
  30. 2025ReMu: Reconstructing Multi-layer 3D Clothed Human from Image Layerscited by 0
  31. 2025SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Videocited by 0
  32. 2026Autoregressive Appearance Prediction for 3D Gaussian Avatarscited by 0
  33. 2026GaussiAnimate: Rig Animatable Categories with Level of Dynamicscited by 0
  34. 2026CLOTH-HUGS: Cloth Aware Human Gaussian Splattingcited by 0
  35. 2026CausalGS: Learning Physical Causality of 3D Dynamic Scenes with Gaussian Representationscited by 0
  36. 2026Latent Dynamics for Full Body Avatar Animationcited by 0
  37. 2026Beyond Thermal Imaging: Inferring Thermophysical Properties from Time-Resolved Thermal Observationscited by 0
  38. 20264DHumanDiff: Direct Text-to-4DGS Generation for Consistent 360-Degree Dynamic Humanscited by 0

50 most cited of 51 All 51 (JSON)