PhysLayer: Language-Guided Layered Animation with Depth-Aware Physics
Not on the map. Why?
Abstract (arXiv)
Existing image-to-video generation methods often produce physically implausible motions and lack precise control over object dynamics. While prior approaches have incorporated physics simulators, they remain confined to 2D planar motions and fail to capture depth-aware spatial interactions. We introduce PhysLayer, a novel framework enabling language-guided, depth-aware layered animation of static images. PhysLayer consists of three key components: First, a language-guided scene understanding module that utilizes vision foundation models to decompose scenes into depth-based layers by analyzing object composition, material properties, and physical parameters. Second, a depth-aware layered physics simulation that extends 2D rigid-body dynamics with depth motion and perspective-consistent scaling, enabling more realistic object interactions without requiring full 3D reconstruction. Third, a physics-guided video synthesis module that integrates simulated trajectories with scene-aware relighting for temporally coherent results. Experimental results demonstrate improvements in CLIP-Similarity (+2.2%), FID score (+9.3%), and Motion-FID (+3%), with human evaluation showing enhanced physical plausibility (+24%) and text-video alignment (+35%). Our approach provides a practical balance between physical realism and computational efficiency for controllable image animation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 29 references that are in the map. First posted on arXiv 26 Apr 2026.
Builds on
23 of 29 references.
References · 23
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
- 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
- 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
- 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 286
- 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 233
11 more
- 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
- 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 165
- 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 154
- 2019PHYRE: A New Benchmark for Physical Reasoningcited by 137
- 2021Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoningcited by 68
- 2020Animating Pictures with Eulerian Motion Fieldscited by 55
- 2023Intrinsic Image Decomposition via Ordinal Shadingcited by 53
- 2024Physical Property Understanding from Language-Embedded Feature Fieldscited by 48
- 2023PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Modelscited by 30
- 2024CLIPAway: Harmonizing Focused Embeddings for Removing Objects via Diffusion Modelscited by 19
- 2024Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Modelcited by 6
Built on by
Not cited yet.