mlsanityData to 7 Oct 2026

InteractPro: A Unified Framework for Motion-Aware Image Composition

arXiv 2409.10090 ↗First posted 16 Sep 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce InteractPro, a comprehensive framework for dynamic motion-aware image composition. At its core is InteractPlan, an intelligent planner that leverages a Large Vision Language Model (LVLM) for scenario analysis and object placement, determining the optimal composition strategy to achieve realistic motion effects. Based on each scenario, InteractPlan selects between our two specialized modules: InteractPhys and InteractMotion. InteractPhys employs an enhanced Material Point Method (MPM)-based simulation to produce physically faithful and controllable object-scene interactions, capturing diverse and abstract events that require true physical modeling. InteractMotion, in contrast, is a training-free method based on pretrained video diffusion. Traditional composition approaches suffer from two major limitations: requiring manual planning for object placement and generating static, motionless outputs. By unifying simulation-based and diffusion-based methods under planner guidance, InteractPro overcomes these challenges, ensuring richly motion-aware compositions. Extensive quantitative and qualitative evaluations demonstrate InteractPro's effectiveness in producing controllable, and coherent compositions across varied scenarios.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 16 Sep 2024.

Builds on

42 of 61 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2023Segment Anythingcited by 8,158
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
30 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  3. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  4. 2023PaLM 2 Technical Reportcited by 1,154
  5. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  6. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  7. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  8. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  9. 2023ModelScope Text-to-Video Technical Reportcited by 651
  10. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  11. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  12. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 368
  13. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  14. 2023GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splattingcited by 331
  15. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  16. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  17. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
  18. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  19. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  20. 2023PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identificationcited by 145
  21. 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 125
  22. 2024Physics-Informed Diffusion Modelscited by 88
  23. 2024Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusioncited by 76
  24. 2023GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planningcited by 61
  25. 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 53
  26. 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
  27. 2024Text-to-Image Rectified Flow as Plug-and-Play Priorscited by 38
  28. 2022Virtual Elastic Objectscited by 35
  29. 2023Cross-domain Compositing with Pretrained Diffusion Modelscited by 13
  30. 2023Effective End-to-End Vision Language Pretraining with Semantic Visual Losscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Interact3D: Compositional 3D Generation of Interactive Objects
  2. 2026ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing
  3. 2026WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
  4. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models
  5. 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
  6. 2024InterDyn: Controllable Interactive Dynamics with Video Diffusion Models
  7. 2025FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors
  8. 2026Alaya-EVOKE: From Linear-Scaling Supervision to Endless World