DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos
On the map: World models → Latent predictive world models
Abstract (arXiv)
World models for deformable objects should recover not only geometry and appearance, but also underlying physical dynamics, interaction grounding, and material behavior. Learning such a model from real videos is challenging because deformable linear, planar, and volumetric objects evolve under high-dimensional deformation, noisy interactions, and complex material response. The model must therefore infer a physical state from visual observations, roll it forward under new interactions, and render the resulting dynamics with high visual fidelity. We present DeformMaster, a video-derived interactive physics-neural world model that turns real interaction videos into an online interactive model of deformable objects within a unified dynamics-and-appearance framework. DeformMaster preserves structured physical rollout while using a neural residual to compensate for unmodeled effects, grounds sparse hand motion as distributed compliant actuator for hand-continuum interaction, represents material response with spatially varying constitutive experts, and drives high-fidelity 4D appearance from the predicted physical evolution. Experiments on real-world deformable-object sequences demonstrate DeformMaster's ability to roll out future dynamics and render dynamic appearance, outperforming state-of-the-art baselines while supporting novel action rollout, material-parameter variation, and dynamic novel-view synthesis. Project page: https://can-lee.github.io/deformmaster-web/
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 32 of its 40 references that are in the map. First posted on arXiv 10 May 2026.
Builds on
32 of 40 references.
Closest ancestors · 3
Same direction, newest first
- 2026Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goals9 Jan
- 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signals26 May
- 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation19 Apr
Other references · 29
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
- 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
- 2023SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenescited by 347
- 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 261
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
- 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
- 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
17 more
- 2024Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussianscited by 97
- 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 87
- 2025OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generationcited by 69
- 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 66
- 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 54
- 2024GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulationcited by 49
- 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 45
- 2024Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modelingcited by 43
- 2024PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulationcited by 36
- 2024Efficient Physics Simulation for 3D Scenes via MLLM-Guided Gaussian Splattingcited by 32
- 2025Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactionscited by 30
- 2025Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videoscited by 25
- 2026RealWonder: Real-Time Physical Action-Conditioned Video Generationcited by 22
- 2025PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesiscited by 10
- 2026EMPM: Embodied MPM for Modeling and Simulation of Deformable Objectscited by 9
- 2026PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generationcited by 9
- 2026Learning Physics-Grounded 4D Dynamics with Neural Gaussian Force Fieldscited by 6
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis
- 2026Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models
- 2026PhysCoRe: Physics-Corrected Residual World Models for Material-Aware Deformable Dynamics
- 2026DeformSmith: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation ↗
- 2024DeformNet: Latent Space Modeling and Dynamics Prediction for Deformable Object Manipulation
- 2026StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions
- 2026Instant-Fold: In-Context Imitation Learning for Deformable Object Manipulation
- 2026SoftVTBench: A Deformation-Aware Visuo-Tactile Dataset and Benchmark for Deformable-Object Manipulation