mlsanityData to 14 Oct 2026

MotionCharacter: Fine-Grained Motion Controllable Human Video Generation

arXiv 2411.18281 ↗First posted 27 Nov 2024cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent advancements in personalized Text-to-Video (T2V) generation have made significant strides in synthesizing character-specific content. However, these methods face a critical limitation: the inability to perform fine-grained control over motion intensity. This limitation stems from an inherent entanglement of action semantics and their corresponding magnitudes within coarse textual descriptions, hindering the generation of nuanced human videos and limiting their applicability in scenarios demanding high precision, such as animating virtual avatars or synthesizing subtle micro-expressions. Furthermore, existing approaches often struggle to preserve high identity fidelity when other attributes are modified. To address these challenges, we introduce MotionCharacter, a framework for high-fidelity human video generation with precise motion control. At its core, MotionCharacter explicitly decouples motion into two independently controllable components: action type and motion intensity. This is achieved through two key technical contributions: (1) a Motion Control Module that leverages textual phrases to specify the action type and a quantifiable metric derived from optical flow to modulate its intensity, guided by a region-aware loss that localizes motion to relevant subject areas; and (2) an ID Content Insertion Module coupled with an ID-Consistency loss to ensure robust identity preservation during dynamic motions. To facilitate training for such fine-grained control, we also curate Human-Motion, a new large-scale dataset with detailed annotations for both motion and facial features. Extensive experiments demonstrate that MotionCharacter achieves substantial improvements over existing methods. Our framework excels in generating videos that are not only identity-consistent but also precisely adhere to specified motion types and intensities.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

28 of 40 references.

References · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  8. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  11. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  12. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
16 more
  1. 2023ModelScope Text-to-Video Technical Reportcited by 658
  2. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  3. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  4. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  5. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
  6. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  7. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
  8. 2023Dreamix: Video Diffusion Models are General Video Editorscited by 191
  9. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 176
  10. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  11. 2024ID-Animator: Zero-Shot Identity-Preserving Human Video Generationcited by 124
  12. 2023CelebV-Text: A Large-Scale Facial Text-Video Datasetcited by 104
  13. 2024CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilitiescited by 60
  14. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 59
  15. 2024LCM-Lookahead for Encoder-based Text-to-Image Personalizationcited by 48
  16. 2021BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generationcited by 47

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.