mlsanityData to 7 Oct 2026

MotionBooth: Motion-Aware Customized Text-to-Video Generation

arXiv 2406.17758 ↗First posted 25 Jun 2024cs.CVcited by 92

Not on the map. Why?

Cited by 34 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we present MotionBooth, an innovative framework designed for animating customized subjects with precise control over both object and camera movements. By leveraging a few images of a specific object, we efficiently fine-tune a text-to-video model to capture the object's shape and attributes accurately. Our approach presents subject region loss and video preservation loss to enhance the subject's learning performance, along with a subject token cross-attention loss to integrate the customized subject with motion control signals. Additionally, we propose training-free techniques for managing subject and camera motions during inference. In particular, we utilize cross-attention map manipulation to govern subject motion and introduce a novel latent shift module for camera movement control as well. MotionBooth excels in preserving the appearance of subjects while simultaneously controlling the motions in generated videos. Extensive quantitative and qualitative evaluations demonstrate the superiority and effectiveness of our method. Our project page is at https://jianzongwu.github.io/projects/motionbooth

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Builds on 64 of its 73 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

64 of 73 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2023Segment Anythingcited by 8,158
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
38 more
  1. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  2. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  5. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  6. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  7. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  8. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  9. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  10. 2023ModelScope Text-to-Video Technical Reportcited by 651
  11. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  12. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  13. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  14. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  15. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  16. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  17. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  18. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  19. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  20. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  21. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  22. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  23. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  24. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  25. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  26. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280
  27. 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
  28. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  29. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  30. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  31. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
  32. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  33. 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
  34. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 138
  35. 2023VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planningcited by 138
  36. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 124
  37. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  38. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 59; 2026: 20.

92 papers.

Sort:

By area:

  1. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  2. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  3. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
  4. 2025Controllable Video Generation: A Surveycited by 93
  5. 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 73
  6. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  7. 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 72
  8. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  9. 2025Towards Understanding Camera Motions in Any Videocited by 62
  10. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 60
  11. 2024Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesiscited by 55
  12. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
38 more
  1. 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generationcited by 50
  2. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 49
  3. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 45
  4. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  5. 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 42
  6. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 42
  7. 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 33
  8. 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 29
  9. 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 27
  10. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 26
  11. 2024SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Devicecited by 25
  12. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  13. 2025DreamRelation: Relation-Centric Video Customizationcited by 23
  14. 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
  15. 2025DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformerscited by 20
  16. 2024AKiRa: Augmentation Kit on Rays for optical video generationcited by 17
  17. 2025VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Modelscited by 17
  18. 2024DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generationcited by 16
  19. 2024OnlyFlow: Optical Flow based Motion Conditioning for Video Diffusion Modelscited by 15
  20. 2025Free-Form Motion Control: Controlling the 6D Poses of Camera and Objects in Video Generationcited by 15
  21. 2025VFX Creator: Animated Visual Effect Generation with Controllable Diffusion Transformercited by 15
  22. 2025SketchVideo: Sketch-based Video Generation and Editingcited by 15
  23. 2025Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfercited by 14
  24. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  25. 2025Ingredients: Blending Custom Photos with Video Diffusion Transformerscited by 13
  26. 2025Tora2: Motion and Appearance Customized Diffusion Transformer for Multi-Entity Video Generationcited by 12
  27. 2025MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agentcited by 11
  28. 2025AnyI2V: Animating Any Conditional Image with Motion Controlcited by 11
  29. 2025AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Modelscited by 10
  30. 2024DreamRelation: Bridging Customization and Relation Generationcited by 9
  31. 2025I2V3D: Controllable image-to-video generation with 3D guidancecited by 9
  32. 2025TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generationcited by 9
  33. 2025DualReal: Adaptive Joint Training for Lossless Identity-Motion Fusion in Video Customizationcited by 9
  34. 2025WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generationcited by 9
  35. 2025DisMo: Disentangled Motion Representations for Open-World Motion Transfercited by 9
  36. 2025SMRABooth: Subject and Motion Representation Alignment for Customized Video Generationcited by 9
  37. 2025Identity-Preserving Text-to-Video Generation via Training-Free Prompt, Image, and Guidance Enhancementcited by 8
  38. 2025Target-Aware Video Diffusion Modelscited by 7

50 most cited of 92 All 92 (JSON)