mlsanityData to 7 Oct 2026

Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision

arXiv 2501.01715 ↗First posted 3 Jan 2025cs.CV cs.ROcited by 7

Not on the map. Why?

Abstract (arXiv)

We introduce Cloth-Splatting, a method for estimating 3D states of cloth from RGB images through a prediction-update framework. Cloth-Splatting leverages an action-conditioned dynamics model for predicting future states and uses 3D Gaussian Splatting to update the predicted states. Our key insight is that coupling a 3D mesh-based representation with Gaussian Splatting allows us to define a differentiable map between the cloth state space and the image space. This enables the use of gradient-based optimization techniques to refine inaccurate state estimates using only RGB supervision. Our experiments demonstrate that Cloth-Splatting not only improves state estimation accuracy over current baselines but also reduces convergence time.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 56 of its 68 references that are in the map. First posted on arXiv 3 Jan 2025.

Builds on

56 of 68 references.

References · 56

  1. 2023Segment Anythingcited by 8,158
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  6. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  7. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  8. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  9. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  10. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  11. 2023CoTracker: It is Better to Track Togethercited by 516
  12. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
38 more
  1. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 368
  2. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  3. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  4. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  5. 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 199
  6. 2023Tracking Everything Everywhere All at Oncecited by 190
  7. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  8. 2024VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Realitycited by 175
  9. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
  10. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 121
  11. 2020BCNet: Learning Body and Cloth Shape from A Single Imagecited by 118
  12. 2020SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulationcited by 118
  13. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  14. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 76
  15. 2020STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Renderingcited by 76
  16. 2024GaMeS: Mesh-Based Adapting and Modification of Gaussian Splattingcited by 69
  17. 2021Deep Regression on Manifolds: A 3D Rotation Case Studycited by 67
  18. 2021Learning Visible Connectivity Dynamics for Cloth Smoothingcited by 61
  19. 2020DeepCloth: Neural Garment Representation for Shape and Style Editingcited by 54
  20. 2022SpeedFolding: Learning Efficient Bimanual Folding of Garmentscited by 51
  21. 2019Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisorcited by 44
  22. 2021Challenges and Outlook in Robotic Manipulation of Deformable Objectscited by 43
  23. 2023DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulationcited by 42
  24. 2021GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completioncited by 41
  25. 2022Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulationcited by 32
  26. 2022DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objectscited by 29
  27. 2024Mesh-based Gaussian Splatting for Real-time Large-scale Deformationcited by 28
  28. 2015Folding Deformable Objects using Predictive Simulation and Trajectory Optimizationcited by 19
  29. 2021VisuoSpatial Foresight for Physical Sequential Fabric Manipulationcited by 18
  30. 2022Learning Deformable Object Manipulation from Expert Demonstrationscited by 18
  31. 2023One Policy to Dress Them All: Learning to Dress People with Diverse Poses and Garmentscited by 18
  32. 2021Learning Latent Graph Dynamics for Visual Manipulation of Deformable Objectscited by 17
  33. 2022DextAIRity: Deformable Manipulation Can be a Breezecited by 17
  34. 2024Unfolding the Literature: A Review of Robotic Cloth Manipulationcited by 14
  35. 2023Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamicscited by 11
  36. 2024Learning Keypoints for Robotic Cloth Manipulation using Synthetic Datacited by 11
  37. 2021Learning Visual Feedback Control for Dynamic Cloth Foldingcited by 10
  38. 2022EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamicscited by 10

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twincited by 11
  2. 2025Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fieldscited by 3
  3. 20263D Point World Models: Point Completion Enables More Accurate Dynamics Learningcited by 1
  4. 2026PhysCoRe: Physics-Corrected Residual World Models for Material-Aware Deformable Dynamicscited by 1
  5. 2025DLO-Splatting: Tracking Deformable Linear Objects Using 3D Gaussian Splattingcited by 0
  6. 2026CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditionscited by 0
  7. 2026MeshPriorDiT: Hierarchical Modeling for Action-Conditioned Cloth Dynamicscited by 0