# Diffusion-based Human Motion Style Transfer with Semantic Guidance arXiv: https://arxiv.org/abs/2405.06646 ยท HTML: https://mlsanity.com/paper/2405.06646/ ## Key facts - Cited by 7 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 40 of its 70 references that are in the map. - First posted on arXiv 20 Mar 2024. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 30 of 40) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017) - [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017) - [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019) - [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018) - [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018) - [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022) - [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022) - [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021) - [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017) - [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023) - [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023) - [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021) - [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022) - [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023) - [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020) - [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022) - [ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model](https://mlsanity.com/paper/2304.01116/) (2023) - [Dual Diffusion Implicit Bridges for Image-to-Image Translation](https://mlsanity.com/paper/2203.08382/) (2022) - [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021) - [TEACH: Temporal Action Composition for 3D Humans](https://mlsanity.com/paper/2209.04066/) (2022) - [GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents](https://mlsanity.com/paper/2303.14613/) (2023) - [StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models](https://mlsanity.com/paper/2308.07863/) (2023) - [AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism](https://mlsanity.com/paper/2309.00796/) (2023) - [Unpaired Motion Style Transfer from Video to Animation](https://mlsanity.com/paper/2005.05751/) (2020) - [QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars](https://mlsanity.com/paper/2209.09391/) (2022) ## Built on by (most cited first, 7 of 7) - [LEAD: Latent Realignment for Human Motion Diffusion](https://mlsanity.com/paper/2410.14508/) (2024) - [HMVLM: Human Motion-Vision-Lanuage Model via MoE LoRA](https://mlsanity.com/paper/2511.01463/) (2025) - [ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation](https://mlsanity.com/paper/2512.02453/) (2025) - [ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos](https://mlsanity.com/paper/2604.10466/) (2026) - [STyMo: Fast and Controllable Few-Shot Motion Style Transfer](https://mlsanity.com/paper/2609.04500/) (2026) - [Flexible Motion Generation from Language and Style References](https://mlsanity.com/paper/2609.08032/) (2026) - [Motion Style Slider: Endpoint-Supervised Continuous Style Control for Human Motion Diffusion](https://mlsanity.com/paper/2609.30795/) (2026) ## Abstract (arXiv) 3D Human motion style transfer is a fundamental problem in computer graphic and animation processing. Existing AdaIN- based methods necessitate datasets with balanced style distribution and content/style labels to train the clustered latent space. However, we may encounter a single unseen style example in practical scenarios, but not in sufficient quantity to constitute a style cluster for AdaIN-based methods. Therefore, in this paper, we propose a novel two-stage framework for few-shot style transfer learning based on the diffusion model. Specifically, in the first stage, we pre-train a diffusion-based text-to-motion model as a generative prior so that it can cope with various content motion inputs. In the second stage, based on the single style example, we fine-tune the pre-trained diffusion model in a few-shot manner to make it capable of style transfer. The key idea is regarding the reverse process of diffusion as a motion-style translation process since the motion styles can be viewed as special motion variations. During the fine-tuning for style transfer, a simple yet effective semantic-guided style transfer loss coordinated with style example reconstruction loss is introduced to supervise the style transfer in CLIP semantic space. The qualitative and quantitative evaluations demonstrate that our method can achieve state-of-the-art performance and has practical applications.