mlsanityData to 7 Oct 2026

Capturing and Animation of Body and Clothing from Monocular Video

arXiv 2210.01868 ↗First posted 4 Oct 2022cs.CV cs.GRcited by 83

Not on the map. Why?

Abstract (arXiv)

While recent work has shown progress on extracting clothed 3D human avatars from a single image, video, or a set of 3D scans, several limitations remain. Most methods use a holistic representation to jointly model the body and clothing, which means that the clothing and body cannot be separated for applications like virtual try-on. Other methods separately model the body and clothing, but they require training from a large set of 3D clothed human meshes obtained from 3D/4D scanners or physics simulations. Our insight is that the body and clothing have different modeling requirements. While the body is well represented by a mesh-based parametric 3D model, implicit representations and neural radiance fields are better suited to capturing the large variety in shape and appearance present in clothing. Building on this insight, we propose SCARF (Segmented Clothed Avatar Radiance Field), a hybrid model combining a mesh-based body with a neural radiance field. Integrating the mesh into the volumetric rendering in combination with a differentiable rasterizer enables us to optimize SCARF directly from monocular videos, without any 3D supervision. The hybrid modeling enables SCARF to (i) animate the clothed body avatar by changing body poses (including hand articulation and facial expressions), (ii) synthesize novel views of the avatar, and (iii) transfer clothing between avatars in virtual try-on applications. We demonstrate that SCARF reconstructs clothing with higher visual quality than existing methods, that the clothing deforms with changing body pose and body shape, and that clothing can be successfully transferred between avatars of different subjects. The code and models are available at https://github.com/YadiraF/SCARF.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). Builds on 61 of its 76 references that are in the map. First posted on arXiv 4 Oct 2022.

Builds on

61 of 76 references.

References · 61

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  3. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  4. 2017End-to-end Recovery of Human Shape and Posecited by 883
  5. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  6. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  7. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  8. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  9. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  10. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  11. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  12. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
38 more
  1. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  2. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  3. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  4. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  5. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  6. 2018Video Based Reconstruction of 3D People Modelscited by 237
  7. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  8. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  9. 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
  10. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  11. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  12. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
  13. 2019DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracingcited by 179
  14. 2021ARCH++: Animation-Ready Clothed Human Reconstruction Revisitedcited by 174
  15. 2019Learning to Reconstruct People in Clothing from a Single RGB Cameracited by 168
  16. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 168
  17. 2019Tex2Shape: Detailed Full Human Body Geometry From a Single Imagecited by 166
  18. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  19. 2019Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesiscited by 161
  20. 2021H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motioncited by 157
  21. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  22. 2021SMPLicit: Topology-aware Generative Model for Clothed Peoplecited by 154
  23. 2020TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Stylecited by 148
  24. 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
  25. 2021Collaborative Regression of Expressive Bodies using Moderationcited by 137
  26. 2020BCNet: Learning Body and Cloth Shape from A Single Imagecited by 118
  27. 2019CLOTH3D: Clothed 3D Humanscited by 114
  28. 2018Detailed Human Avatars from Monocular Videocited by 109
  29. 2021Animatable Neural Radiance Fields from Monocular RGB Videoscited by 109
  30. 2021LEAP: Learning Articulated Occupancy of Peoplecited by 107
  31. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  32. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  33. 2021imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Posecited by 102
  34. 2020Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Imagescited by 101
  35. 2019Learning-Based Animation of Clothing for Virtual Try-Oncited by 89
  36. 2020SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothingcited by 87
  37. 2018Image Inpainting via Generative Multi-column Convolutional Neural Networkscited by 81
  38. 2019360-Degree Textures of People in Clothing from a Single Imagecited by 75

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 31; 2024: 31; 2025: 11; 2026: 8.

83 papers.

Sort:

By area:

  1. 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
  2. 2024SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splattingcited by 171
  3. 2023HUGS: Human Gaussian Splatscited by 154
  4. 2022InstantAvatar: Learning Avatars from Monocular Video in 60 Secondscited by 138
  5. 2023Drivable 3D Gaussian Avatarscited by 120
  6. 2023TADA! Text to Animatable Digital Avatarscited by 109
  7. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  8. 2023ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Renderingcited by 104
  9. 2023GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphscited by 96
  10. 2023AG3D: Learning to Generate 3D Avatars from 2D Image Collectionscited by 55
  11. 2024PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observationscited by 51
  12. 2023PoseVocab: Learning Joint-structured Pose Embeddings for Human Avatar Modelingcited by 50
38 more
  1. 2023Learning Disentangled Avatars with Hybrid 3D Representationscited by 44
  2. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  3. 2024GaussianBody: Clothed Human Reconstruction via 3d Gaussian Splattingcited by 36
  4. 2023SplatArmor: Articulated Gaussian splatting for animatable humans from monocular RGB videoscited by 35
  5. 2024MeshAvatar: Learning High-quality Triangular Human Avatars from Multi-view Videoscited by 34
  6. 2023Disentangled Clothed Avatar Generation from Text Descriptionscited by 33
  7. 2023Animatable 3D Gaussian: Fast and High-Quality Reconstruction of Multiple Human Avatarscited by 30
  8. 2023REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videoscited by 29
  9. 2023Animatable and Relightable Gaussians for High-fidelity Human Avatar Modelingcited by 27
  10. 2024Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Videocited by 25
  11. 2023HumanLiff: Layer-wise 3D Human Generation with Diffusion Modelcited by 23
  12. 2023Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Inputcited by 22
  13. 2024LayGA: Layered Gaussian Avatars for Animatable Clothing Transfercited by 22
  14. 2023NSF: Neural Surface Fields for Human Modeling from Monocular Depthcited by 21
  15. 2024GALA: Generating Animatable Layered Assets from a Single Scancited by 21
  16. 2023AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars Using 2D Diffusioncited by 20
  17. 2023CaPhy: Capturing Physical Properties for Animatable Human Avatarscited by 20
  18. 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracingcited by 20
  19. 2024TELA: Text to Layer-wise 3D Clothed Human Generationcited by 20
  20. 2024ReLoo: Reconstructing Humans Dressed in Loose Garments from Monocular Video in the Wildcited by 19
  21. 2023NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objectscited by 17
  22. 2024UV Gaussians: Joint Learning of Mesh Deformation and Gaussian Textures for Human Avatar Modelingcited by 17
  23. 2024Disco4D: Disentangled 4D Human Generation and Animation from a Single Imagecited by 17
  24. 2024PuzzleAvatar: Assembling 3D Avatars from Personal Albumscited by 16
  25. 2024PERSE: Personalized 3D Generative Avatars from A Single Portraitcited by 15
  26. 2023Rendering Humans from Object-Occluded Monocular Videoscited by 14
  27. 2024En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Datacited by 14
  28. 2024TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generationcited by 14
  29. 2024GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splattingcited by 13
  30. 2024MultiPly: Reconstruction of Multiple People from Monocular Video in the Wildcited by 12
  31. 2023High-Quality Facial Geometry and Appearance Capture at Homecited by 9
  32. 2024Neural-ABC: Neural Parametric Models for Articulated Body with Clothescited by 9
  33. 2024PICA: Physics-Integrated Clothed Avatarcited by 8
  34. 2024PEGASUS: Personalized Generative 3D Avatars with Composable Attributescited by 7
  35. 2024LAGA: Layered 3D Avatar Generation and Customization via Gaussian Splattingcited by 7
  36. 2024HumanCoser: Layered 3D Human Generation via Semantic-Aware Diffusion Modelcited by 7
  37. 2024Real-time Free-view Human Rendering from Sparse-view RGB Videos using Double Unprojected Texturescited by 7
  38. 2025Disentangled Clothed Avatar Generation with Layered Representationcited by 7

50 most cited of 83 All 83 (JSON)