# SMooDi: Stylized Motion Diffusion Model

arXiv: https://arxiv.org/abs/2407.12783 · HTML: https://mlsanity.com/paper/2407.12783/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 53 of its 63 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 53)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model](https://mlsanity.com/paper/2303.09833/) (2023)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [OmniControl: Control Any Joint at Any Time for Human Motion Generation](https://mlsanity.com/paper/2310.08580/) (2023)
- [MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis](https://mlsanity.com/paper/2212.04495/) (2022)
- [ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs](https://mlsanity.com/paper/2311.13600/) (2023)
- [Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models](https://mlsanity.com/paper/2211.09707/) (2022)
- [InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion](https://mlsanity.com/paper/2308.16905/) (2023)
- [MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model](https://mlsanity.com/paper/2404.19759/) (2024)
- [GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents](https://mlsanity.com/paper/2303.14613/) (2023)
- [MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators](https://mlsanity.com/paper/2306.10900/) (2023)
- [Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion](https://mlsanity.com/paper/2304.01893/) (2023)
- [MMM: Generative Masked Motion Model](https://mlsanity.com/paper/2312.03596/) (2023)
- [Optimizing Diffusion Noise Can Serve As Universal Motion Priors](https://mlsanity.com/paper/2312.11994/) (2023)
- [Unpaired Motion Style Transfer from Video to Animation](https://mlsanity.com/paper/2005.05751/) (2020)

## Built on by (most cited first, 30 of 37)

- [Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression](https://mlsanity.com/paper/2411.16575/) (2024)
- [InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2502.20390/) (2025)
- [ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model](https://mlsanity.com/paper/2412.14559/) (2024)
- [MaskControl: Spatio-Temporal Control for Masked Motion Synthesis](https://mlsanity.com/paper/2410.10780/) (2024)
- [MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm](https://mlsanity.com/paper/2502.02358/) (2025)
- [Absolute Coordinates Make Motion Generation Easy](https://mlsanity.com/paper/2505.19377/) (2025)
- [A Survey on Human Interaction Motion Generation](https://mlsanity.com/paper/2503.12763/) (2025)
- [Pay Attention and Move Better: Harnessing Attention for Interactive Motion Generation and Training-free Editing](https://mlsanity.com/paper/2410.18977/) (2024)
- [Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusion](https://mlsanity.com/paper/2503.19557/) (2025)
- [SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis](https://mlsanity.com/paper/2412.20104/) (2024)
- [PersonaBooth: Personalized Text-to-Motion Generation](https://mlsanity.com/paper/2503.07390/) (2025)
- [PRIMAL: Physically Reactive and Interactive Motor Model for Avatar Learning](https://mlsanity.com/paper/2503.17544/) (2025)
- [Motion Generation: A Survey of Generative Approaches and Benchmarks](https://mlsanity.com/paper/2507.05419/) (2025)
- [StyleMotif: Multi-Modal Motion Stylization using Style-Content Cross Fusion](https://mlsanity.com/paper/2503.21775/) (2025)
- [ProjFlow: Projection Sampling with Flow Matching for Zero-Shot Exact Spatial Motion Control](https://mlsanity.com/paper/2602.22742/) (2026)
- [VQ-Style: Disentangling Style and Content in Motion with Residual Quantized Representations](https://mlsanity.com/paper/2602.02334/) (2026)
- [MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow](https://mlsanity.com/paper/2412.09901/) (2024)
- [ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis](https://mlsanity.com/paper/2503.06499/) (2025)
- [RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generation](https://mlsanity.com/paper/2605.26241/) (2026)
- [ACMo: Attribute Controllable Motion Generation](https://mlsanity.com/paper/2503.11038/) (2025)
- [RESCUE: Crowd Evacuation Simulation via Controlling SDM-United Characters](https://mlsanity.com/paper/2507.20117/) (2025)
- [ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation](https://mlsanity.com/paper/2512.02453/) (2025)
- [Controllable Long-term Motion Generation with Extended Joint Targets](https://mlsanity.com/paper/2512.04487/) (2025)
- [Sketch2Colab: Sketch-Conditioned Multi-Human Animation via Controllable Flow Distillation](https://mlsanity.com/paper/2603.02190/) (2026)
- [IAM: Identity-Aware Human Motion and Shape Joint Generation](https://mlsanity.com/paper/2604.25164/) (2026)
- [Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation](https://mlsanity.com/paper/2607.06843/) (2026)
- [LILAC: Long-sequence Incremental Low-latency Arbitrary Motion Stylization via Streaming VAE-Diffusion with Causal Decoding](https://mlsanity.com/paper/2510.15392/) (2025)
- [One-shot Adaptation of Humanoid Whole-body Motion with Walking Priors](https://mlsanity.com/paper/2510.25241/) (2025)
- [Robo-Saber: Generating and Simulating Virtual Reality Players](https://mlsanity.com/paper/2602.18319/) (2026)
- [Multi-scale Coarse-to-fine Modeling for Test-time Human Motion Control](https://mlsanity.com/paper/2605.14935/) (2026)

## Abstract (arXiv)

We introduce a novel Stylized Motion Diffusion model, dubbed SMooDi, to generate stylized motion driven by content texts and style motion sequences. Unlike existing methods that either generate motion of various content or transfer style from one sequence to another, SMooDi can rapidly generate motion across a broad range of content and diverse styles. To this end, we tailor a pre-trained text-to-motion model for stylization. Specifically, we propose style guidance to ensure that the generated motion closely matches the reference style, alongside a lightweight style adaptor that directs the motion towards the desired style while ensuring realism. Experiments across various applications demonstrate that our proposed framework outperforms existing methods in stylized motion generation.