mlsanityData to 14 Oct 2026

SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video

arXiv 2509.08828 ↗First posted 10 Sep 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The reconstruction of three-dimensional dynamic scenes is a well-established yet challenging task within the domain of computer vision. In this paper, we propose a novel approach that combines the domains of 3D geometry reconstruction and appearance estimation for physically based rendering and present a system that is able to perform both tasks for fabrics, utilizing only a single monocular RGB video sequence as input. In order to obtain realistic and high-quality deformations and renderings, a physical simulation of the cloth geometry and differentiable rendering are employed. In this paper, we introduce two novel regularization terms for the 3D reconstruction task that improve the plausibility of the reconstruction by addressing the depth ambiguity problem in monocular video. In comparison with the most recent methods in the field, we have reduced the error in the 3D reconstruction by a factor of 2.64 while requiring a medium runtime of 30 min per scene. Furthermore, the optimized motion achieves sufficient quality to perform an appearance estimation of the deforming object, recovering sharp details from this single monocular RGB video.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 82 references that are in the map. First posted on arXiv 10 Sep 2025.

Builds on

47 of 82 references.

References · 47

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023Segment Anythingcited by 8,234
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  5. 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
  6. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  7. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  8. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  9. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  10. 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 258
  11. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  12. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
35 more
  1. 2019Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networkscited by 129
  2. 2018Single-Image SVBRDF Capture with a Rendering-Aware Deep Networkcited by 116
  3. 2022IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Imagescited by 111
  4. 20233D Human Pose Estimation via Intuitive Physicscited by 103
  5. 2022SNUG: Self-Supervised Neural Dynamic Garmentscited by 83
  6. 2022HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamicscited by 78
  7. 2020Scalable Differentiable Physics for Learning and Controlcited by 73
  8. 2021DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contactcited by 65
  9. 2020PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformationcited by 63
  10. 2023Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstructioncited by 61
  11. 2020Two-shot Spatially-varying BRDF and Shape Estimationcited by 59
  12. 2018Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networkscited by 51
  13. 2024PhysAvatar: Learning the Physics of Dressed 3D Avatars from Visual Observationscited by 51
  14. 2022DrapeNet: Garment Generation and Self-Supervised Drapingcited by 41
  15. 2021Shape and Material Capture at Homecited by 36
  16. 2022Neural Cloth Simulationcited by 36
  17. 2022DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objectscited by 31
  18. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  19. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  20. 2024Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Videocited by 25
  21. 2022φ-SfT: Shape-from-Template with a Physics-Based Deformation Modelcited by 23
  22. 2023CaPhy: Capturing Physical Properties for Animatable Human Avatarscited by 20
  23. 2019IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstructioncited by 16
  24. 2023Towards Multi-Layered 3D Garments Animationcited by 16
  25. 2018HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Modelcited by 15
  26. 2018Geometry-Aware Network for Non-Rigid Shape Prediction from a Single Viewcited by 15
  27. 2020AutoClip: Adaptive Gradient Clipping for Source Separation Networkscited by 12
  28. 2022Fine-grained differentiable physics: a yarn-level model for fabricscited by 9
  29. 2023Physics-guided Shape-from-Template: Monocular Video Perception through Neural Surrogate Modelscited by 8
  30. 2024Metamizer: a versatile neural optimizer for fast and accurate physics simulationscited by 7
  31. 2017Better Together: Joint Reasoning for Non-rigid 3D Reconstruction with Specularities and Shadingcited by 6
  32. 2020Learning Incompressible Fluid Dynamics from Scratch -- Towards Fast, Differentiable Fluid Models that Generalizecited by 6
  33. 2023ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Librarycited by 6
  34. 2023How Will It Drape Like? Capturing Fabric Mechanics from Depth Imagescited by 6
  35. 2025ROSA: Reconstructing Object Shape and Appearance Textures by Adaptive Detail Transfercited by 2

Built on by

Not cited yet.