mlsanityData to 7 Oct 2026

Generative Image Dynamics

arXiv 2309.07906 ↗First posted 14 Sep 2023cs.CVcited by 93

Not on the map. Why?

Cited by 26 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present an approach to modeling an image-space prior on scene motion. Our prior is learned from a collection of motion trajectories extracted from real video sequences depicting natural, oscillatory dynamics such as trees, flowers, candles, and clothes swaying in the wind. We model this dense, long-term motion prior in the Fourier domain:given a single image, our trained model uses a frequency-coordinated diffusion sampling process to predict a spectral volume, which can be converted into a motion texture that spans an entire video. Along with an image-based rendering module, these trajectories can be used for a number of downstream applications, such as turning still images into seamlessly looping videos, or allowing users to realistically interact with objects in real pictures by interpreting the spectral volumes as image-space modal bases, which approximate object dynamics.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 85 of its 123 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

85 of 123 references.

References · 85

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  12. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
38 more
  1. 2017The Kinetics Human Action Video Datasetcited by 1,894
  2. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  5. 2018Demystifying MMD GANscited by 1,268
  6. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  8. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  9. 2022Human Motion Diffusion Modelcited by 1,062
  10. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  11. 2020First Order Motion Model for Image Animationcited by 698
  12. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  13. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  14. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  15. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  16. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  17. 2023Universal Guidance for Diffusion Modelscited by 426
  18. 2016Generating Videos with Scene Dynamicscited by 376
  19. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  20. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  21. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  22. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  23. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  24. 2022Novel View Synthesis with Diffusion Modelscited by 299
  25. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
  26. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  27. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  28. 2021Motion Representations for Articulated Animationcited by 253
  29. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  30. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  31. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  32. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
  33. 2022Thin-Plate Spline Motion Model for Image Animationcited by 209
  34. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 206
  35. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  36. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  37. 2018Stochastic Adversarial Video Predictioncited by 173
  38. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 33; 2025: 35; 2026: 15.

93 papers.

Sort:

By area:

  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  2. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  3. 2023A Survey on Video Diffusion Modelscited by 203
  4. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  5. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
  6. 2023FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editingcited by 165
  7. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  8. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  9. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
  10. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
  11. 2024Video as the New Language for Real-World Decision Makingcited by 86
  12. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
38 more
  1. 2023AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidancecited by 66
  2. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 65
  3. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  4. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52
  5. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 49
  6. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  7. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  8. 2023LivePhoto: Real Image Animation with Text-guided Motion Controlcited by 44
  9. 2024GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modelingcited by 44
  10. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  11. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 43
  12. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 41
  13. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  14. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 40
  15. 2024Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimatorscited by 39
  16. 2025FloVD: Optical Flow Meets Video Diffusion Model for Enhanced Camera-Controlled Video Synthesiscited by 37
  17. 2024RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Modelscited by 36
  18. 2024DragAPart: Learning a Part-Level Motion Prior for Articulated Objectscited by 32
  19. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 32
  20. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  21. 2023PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Modelscited by 30
  22. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  23. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  24. 2024AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AIcited by 27
  25. 2025Generative Physical AI in Vision: A Surveycited by 24
  26. 2023Mini-DALLE3: Interactive Text to Image by Prompting Large Language Modelscited by 22
  27. 2025Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixelscited by 22
  28. 2023MagDiff: Multi-Alignment Diffusion for High-Fidelity Video Generation and Editingcited by 21
  29. 2024Explorative Inbetweening of Time and Spacecited by 21
  30. 2024VividDream: Generating 3D Scene with Ambient Dynamicscited by 21
  31. 2024FrameBridge: Improving Image-to-Video Generation with Bridge Modelscited by 19
  32. 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
  33. 2025PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Modelcited by 15
  34. 2023Event-based Continuous Color Video Decompression from Single Framescited by 14
  35. 2024Generative Visual Compression: A Reviewcited by 13
  36. 2024VideoElevator: Elevating Video Generation Quality with Versatile Text-to-Image Diffusion Modelscited by 13
  37. 2024SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesiscited by 12
  38. 2024Motion Modes: What Could Happen Next?cited by 11

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Controllable Longer Image Animation with Diffusion Models
  2. 2024Diffusion Priors for Dynamic View Synthesis from Monocular Videos
  3. 2026TeDiO: Temporal Diagonal Optimization for Training-Free Coherent Video Diffusion
  4. 2022Video Diffusion Models
  5. 2026DynaVid: Learning to Generate Highly Dynamic Videos using Synthetic Motion Data
  6. 2024Video Generation with Learned Action Prior
  7. 2023LEO: Generative Latent Image Animator for Human Video Synthesis
  8. 2026Spectral Progressive Diffusion for Efficient Image and Video Generation