InteractPro: A Unified Framework for Motion-Aware Image Composition
Not on the map. Why?
Abstract (arXiv)
We introduce InteractPro, a comprehensive framework for dynamic motion-aware image composition. At its core is InteractPlan, an intelligent planner that leverages a Large Vision Language Model (LVLM) for scenario analysis and object placement, determining the optimal composition strategy to achieve realistic motion effects. Based on each scenario, InteractPlan selects between our two specialized modules: InteractPhys and InteractMotion. InteractPhys employs an enhanced Material Point Method (MPM)-based simulation to produce physically faithful and controllable object-scene interactions, capturing diverse and abstract events that require true physical modeling. InteractMotion, in contrast, is a training-free method based on pretrained video diffusion. Traditional composition approaches suffer from two major limitations: requiring manual planning for object placement and generating static, motionless outputs. By unifying simulation-based and diffusion-based methods under planner guidance, InteractPro overcomes these challenges, ensuring richly motion-aware compositions. Extensive quantitative and qualitative evaluations demonstrate InteractPro's effectiveness in producing controllable, and coherent compositions across varied scenarios.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 16 Sep 2024.
Builds on
42 of 61 references.
References · 42
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2023Segment Anythingcited by 8,158
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
30 more
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2023PaLM 2 Technical Reportcited by 1,154
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
- 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
- 2023ModelScope Text-to-Video Technical Reportcited by 651
- 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
- 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
- 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 368
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 331
- 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
- 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
- 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
- 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
- 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 145
- 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 125
- 2024Physics-Informed Diffusion Modelscited by 88
- 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 76
- 2023GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planningcited by 61
- 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 53
- 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
- 2024Text-to-Image Rectified Flow as Plug-and-Play Priorscited by 38
- 2022Virtual Elastic Objectscited by 35
- 2023Cross-domain Compositing with Pretrained Diffusion Modelscited by 13
- 2023Effective End-to-End Vision Language Pretraining with Semantic Visual Losscited by 10
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Interact3D: Compositional 3D Generation of Interactive Objects
- 2026ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing
- 2026WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
- 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models
- 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
- 2024InterDyn: Controllable Interactive Dynamics with Video Diffusion Models
- 2025FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors
- 2026Alaya-EVOKE: From Linear-Scaling Supervision to Endless World