mlsanityData to 7 Oct 2026

Make Pixels Dance: High-Dynamic Video Generation

arXiv 2311.10982 ↗First posted 18 Nov 2023cs.CVcited by 140

Not on the map. Why?

Cited by 52 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Creating high-dynamic videos such as motion-rich actions and sophisticated visual effects poses a significant challenge in the field of artificial intelligence. Unfortunately, current state-of-the-art video generation methods, primarily focusing on text-to-video generation, tend to produce video clips with minimal motions despite maintaining high fidelity. We argue that relying solely on text instructions is insufficient and suboptimal for video generation. In this paper, we introduce PixelDance, a novel approach based on diffusion models that incorporates image instructions for both the first and last frames in conjunction with text instructions for video generation. Comprehensive experimental results demonstrate that PixelDance trained with public data exhibits significantly better proficiency in synthesizing videos with complex scenes and intricate motions, setting a new standard for video generation.

Key facts. Cited by 140 papers in the map (data through 7 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 18 Nov 2023.

Builds on

48 of 58 references.

References · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  9. 2022Video Diffusion Modelscited by 2,280
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
36 more
  1. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  2. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  3. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  4. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  5. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  6. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  7. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  8. 2023ModelScope Text-to-Video Technical Reportcited by 651
  9. 2015Holistically-Nested Edge Detectioncited by 590
  10. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  11. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  12. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  13. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  14. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  15. 2016Generating Videos with Scene Dynamicscited by 376
  16. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  17. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  18. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  19. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  20. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  21. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  22. 2023Dreamix: Video Diffusion Models are General Video Editorscited by 189
  23. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 170
  24. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  25. 2017Video Generation From Textcited by 153
  26. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  27. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generationcited by 137
  28. 2018Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GANcited by 120
  29. 2014Video (language) modeling: a baseline for generative models of natural videoscited by 92
  30. 2018To Create What You Tell: Generating Videos from Captionscited by 84
  31. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  32. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generationcited by 74
  33. 2023Seer: Language Instructed Video Prediction with Latent Diffusion Modelscited by 51
  34. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 49
  35. 2023Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolutioncited by 33
  36. 2023MagicAvatar: Multimodal Avatar Generation and Animationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 58; 2025: 65; 2026: 13.

140 papers.

Sort:

By area:

  1. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  2. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  3. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  4. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 332
  5. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 242
  6. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  7. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 142
  8. 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 132
  9. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  10. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  11. 2025Phantom: Subject-consistent video generation via cross-modal alignmentcited by 126
  12. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
38 more
  1. 2024Training-free Camera Control for Video Generationcited by 119
  2. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  3. 2025Long Context Tuning for Video Generationcited by 110
  4. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  5. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  6. 2024Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenizationcited by 98
  7. 2025Controllable Video Generation: A Surveycited by 93
  8. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restorationcited by 89
  9. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  10. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 87
  11. 2024DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmarkcited by 85
  12. 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 76
  13. 2023I2V-Adapter: A General Image-to-Video Adapter for Diffusion Modelscited by 70
  14. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 70
  15. 2024ToonCrafter: Generative Cartoon Interpolationcited by 68
  16. 2025Goku: Flow Based Video Generative Foundation Modelscited by 67
  17. 2025RewardDance: Reward Scaling in Visual Generationcited by 63
  18. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  19. 2025AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusioncited by 53
  20. 2025InfinityStar: Unified Spacetime AutoRegressive Modeling for Visual Generationcited by 43
  21. 2024Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolationcited by 42
  22. 2024ARLON: Boosting Diffusion Transformers with Autoregressive Models for Long Video Generationcited by 41
  23. 2024A Survey on Long Video Generation: Challenges, Methods, and Prospectscited by 40
  24. 2024TrackGo: A Flexible and Efficient Method for Controllable Video Generationcited by 39
  25. 2024I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strengthcited by 38
  26. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 38
  27. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 34
  28. 2024ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Modelscited by 33
  29. 2025VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual interventioncited by 33
  30. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
  31. 2025MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformerscited by 32
  32. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  33. 2025OneStory: Coherent Multi-Shot Video Generation with Adaptive Memorycited by 31
  34. 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 29
  35. 2024Video Diffusion Models: A Surveycited by 28
  36. 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
  37. 2023Motion-Conditioned Image Animation for Video Editingcited by 26
  38. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 26

50 most cited of 140 All 140 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024ViMo: Generating Motions from Casual Videos
  2. 2024Motion Prompting: Controlling Video Generation with Motion Trajectories
  3. 2024Animate Your Motion: Turning Still Images into Dynamic Videos
  4. 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising
  5. 2023MoVideo: Motion-Aware Video Generation with Diffusion Models
  6. 2023DreaMoving: A Human Video Generation Framework based on Diffusion Models
  7. 2025LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer
  8. 2024Move-in-2D: 2D-Conditioned Human Motion Generation