# SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Prediction

arXiv: https://arxiv.org/abs/2503.18211 · HTML: https://mlsanity.com/paper/2503.18211/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 70 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Builds on (most cited first, 30 of 54)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model](https://mlsanity.com/paper/2304.01116/) (2023)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [OmniControl: Control Any Joint at Any Time for Human Motion Generation](https://mlsanity.com/paper/2310.08580/) (2023)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis](https://mlsanity.com/paper/2212.04495/) (2022)
- [FLAME: Free-form Language-based Motion Synthesis & Editing](https://mlsanity.com/paper/2209.00349/) (2022)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion](https://mlsanity.com/paper/2308.16905/) (2023)

## Built on by (most cited first, 27 of 27)

- [DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling](https://mlsanity.com/paper/2506.18680/) (2025)
- [UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors](https://mlsanity.com/paper/2603.15975/) (2026)
- [Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learning](https://mlsanity.com/paper/2605.30969/) (2026)
- [MDD: A Dataset for Text-and-Music Conditioned Duet Dance Generation](https://mlsanity.com/paper/2508.16911/) (2025)
- [ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation](https://mlsanity.com/paper/2605.11704/) (2026)
- [SceMoS: Scene-Aware 3D Human Motion Synthesis by Planning with Geometry-Grounded Tokens](https://mlsanity.com/paper/2602.20476/) (2026)
- [Cross-Axis Feature Fusion with Joint-Wise Motion Difference Prediction for Text-Based 3D Human Motion Editing](https://mlsanity.com/paper/2606.01014/) (2026)
- [U-Mind: A Unified Framework for Real-Time Multimodal Interaction with Audiovisual Generation](https://mlsanity.com/paper/2602.23739/) (2026)
- [OmniMoGen: Unifying Human Motion Generation via Learning from Interleaved Text-Motion Instructions](https://mlsanity.com/paper/2512.19159/) (2025)
- [Coordinate-Based Dual-Constrained Autoregressive Motion Generation](https://mlsanity.com/paper/2604.08088/) (2026)
- [ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos](https://mlsanity.com/paper/2604.10466/) (2026)
- [A Unified Conditional Flow for Motion Generation, Editing, and Intra-Structural Retargeting](https://mlsanity.com/paper/2604.13427/) (2026)
- [Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editing](https://mlsanity.com/paper/2608.16008/) (2026)
- [FashionPose: Unified Text-Driven Fashion Synthesis with Joint Geometric and Photometric Control](https://mlsanity.com/paper/2507.13311/) (2025)
- [InterEdit: Navigating Text-Guided 3D Dyadic Human Motion Editing](https://mlsanity.com/paper/2603.13082/) (2026)
- [Controllable Text-to-Motion Generation via Modular Body-Part Phase Control](https://mlsanity.com/paper/2603.19795/) (2026)
- [MaskAdapt: Learning Flexible Motion Adaptation via Mask-Invariant Prior for Physics-Based Characters](https://mlsanity.com/paper/2603.29272/) (2026)
- [Towards Highly-Constrained Human Motion Generation with Retrieval-Guided Diffusion Noise Optimization](https://mlsanity.com/paper/2605.08054/) (2026)
- [MotionMERGE: A Multi-granular Framework for Human Motion Editing, Reasoning, Generation, and Explanation](https://mlsanity.com/paper/2605.18956/) (2026)
- [NaP-Control: Navigating Diffusion Prior for Versatile and Fast Character Control](https://mlsanity.com/paper/2605.20209/) (2026)
- [PoseShield: Neural Collision Fields for Human Self-Collision Resolution](https://mlsanity.com/paper/2606.29686/) (2026)
- [Training-free Controllable Human Motion Generation under Heterogeneous Constraints](https://mlsanity.com/paper/2607.01990/) (2026)
- [MIME: Multimodal Interactive Motion Encoder](https://mlsanity.com/paper/2607.22702/) (2026)
- [MoSAIC: Aligned Intervention Supervision for Part-Local Motion Style Transfer](https://mlsanity.com/paper/2607.26304/) (2026)
- [UniMoFlow: Grounding Instruction-Driven 3D Human Motion Editing in Generation](https://mlsanity.com/paper/2608.09143/) (2026)
- [SuperMotion: Source-Preserving Denoising for Text-Driven Human Motion Editing](https://mlsanity.com/paper/2610.01517/) (2026)
- [TAME:Topology-Aware Text-Driven Motion Editing across Heterogeneous Humanoid Skeletons](https://mlsanity.com/paper/2610.04529/) (2026)

## Abstract (arXiv)

Text-based 3D human motion editing is a critical yet challenging task in computer vision and graphics. While training-free approaches have been explored, the recent release of the MotionFix dataset, which includes source-text-motion triplets, has opened new avenues for training, yielding promising results. However, existing methods struggle with precise control, often leading to misalignment between motion semantics and language instructions. In this paper, we introduce a related task, motion similarity prediction, and propose a multi-task training paradigm, where we train the model jointly on motion editing and motion similarity prediction to foster the learning of semantically meaningful representations. To complement this task, we design an advanced Diffusion-Transformer-based architecture that separately handles motion similarity prediction and motion editing. Extensive experiments demonstrate the state-of-the-art performance of our approach in both editing alignment and fidelity.