# Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry

arXiv: https://arxiv.org/abs/2303.08658 · HTML: https://mlsanity.com/paper/2303.08658/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [Deep Convolutional Inverse Graphics Network](https://mlsanity.com/paper/1503.03167/) (2015)
- [Contact and Human Dynamics from Monocular Video](https://mlsanity.com/paper/2007.11678/) (2020)
- [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018)
- [Contact-Aware Retargeting of Skinned Motion](https://mlsanity.com/paper/2109.07431/) (2021)
- [A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos](https://mlsanity.com/paper/2103.15596/) (2021)

## Built on by (most cited first, 30 of 35)

- [TLControl: Trajectory and Language Control for Human Motion Synthesis](https://mlsanity.com/paper/2311.17135/) (2023)
- [EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation](https://mlsanity.com/paper/2312.02256/) (2023)
- [OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers](https://mlsanity.com/paper/2312.08985/) (2023)
- [AnyTop: Character Animation Diffusion with Any Topology](https://mlsanity.com/paper/2502.17327/) (2025)
- [HUMOS: Human Motion Model Conditioned on Body Shape](https://mlsanity.com/paper/2409.03944/) (2024)
- [Motion2Motion: Cross-topology Motion Transfer with Sparse Correspondence](https://mlsanity.com/paper/2508.13139/) (2025)
- [STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargeting](https://mlsanity.com/paper/2406.04629/) (2024)
- [Skinned Motion Retargeting with Dense Geometric Interaction Perception](https://mlsanity.com/paper/2410.20986/) (2024)
- [ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space](https://mlsanity.com/paper/2309.05310/) (2023)
- [Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions](https://mlsanity.com/paper/2504.03639/) (2025)
- [Generative Motion Stylization of Cross-structure Characters within Canonical Motion Space](https://mlsanity.com/paper/2403.11469/) (2024)
- [Pose-to-Motion: Cross-Domain Motion Retargeting with Pose Prior](https://mlsanity.com/paper/2310.20249/) (2023)
- [Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interaction](https://mlsanity.com/paper/2402.04768/) (2024)
- [Semantics-aware Motion Retargeting with Vision-Language Models](https://mlsanity.com/paper/2312.01964/) (2023)
- [SMF: Template-free and Rig-free Animation Transfer using Kinetic Codes](https://mlsanity.com/paper/2504.04831/) (2025)
- [Motion Keyframe Interpolation for Any Human Skeleton via Temporally Consistent Point Cloud Sampling and Reconstruction](https://mlsanity.com/paper/2405.07444/) (2024)
- [I-CTRL: Imitation to Control Humanoid Robots Through Constrained Reinforcement Learning](https://mlsanity.com/paper/2405.08726/) (2024)
- [Towards High-Quality 3D Motion Transfer with Realistic Apparel Animation](https://mlsanity.com/paper/2407.11266/) (2024)
- [Towards Robust 3D Pose Transfer with Adversarial Learning](https://mlsanity.com/paper/2404.02242/) (2024)
- [PALUM: Part-based Attention Learning for Unified Motion Retargeting](https://mlsanity.com/paper/2601.07272/) (2026)
- [ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes](https://mlsanity.com/paper/2604.17623/) (2026)
- [AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance](https://mlsanity.com/paper/2502.08189/) (2025)
- [ASMR: Adaptive Skeleton-Mesh Rigging and Skinning via 2D Generative Prior](https://mlsanity.com/paper/2503.13579/) (2025)
- [STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints](https://mlsanity.com/paper/2504.06504/) (2025)
- [MoReFlow: Motion Retargeting Learning through Unsupervised Flow Matching](https://mlsanity.com/paper/2509.25600/) (2025)
- [ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting](https://mlsanity.com/paper/2605.06593/) (2026)
- [Quality assessment of 3D human animation: Subjective and objective evaluation](https://mlsanity.com/paper/2505.23301/) (2025)
- [Behave Your Motion: Habit-preserved Cross-category Animal Motion Transfer](https://mlsanity.com/paper/2507.07394/) (2025)
- [Gaussian See, Gaussian Do: Semantic 3D Motion Transfer from Multiview Video](https://mlsanity.com/paper/2511.14848/) (2025)
- [A Unified Conditional Flow for Motion Generation, Editing, and Intra-Structural Retargeting](https://mlsanity.com/paper/2604.13427/) (2026)

## Abstract (arXiv)

A good motion retargeting cannot be reached without reasonable consideration of source-target differences on both the skeleton and shape geometry levels. In this work, we propose a novel Residual RETargeting network (R2ET) structure, which relies on two neural modification modules, to adjust the source motions to fit the target skeletons and shapes progressively. In particular, a skeleton-aware module is introduced to preserve the source motion semantics. A shape-aware module is designed to perceive the geometries of target characters to reduce interpenetration and contact-missing. Driven by our explored distance-based losses that explicitly model the motion semantics and geometry, these two modules can learn residual motion modifications on the source motion to generate plausible retargeted motion in a single inference without post-processing. To balance these two modifications, we further present a balancing gate to conduct linear interpolation between them. Extensive experiments on the public dataset Mixamo demonstrate that our R2ET achieves the state-of-the-art performance, and provides a good balance between the preservation of motion semantics as well as the attenuation of interpenetration and contact-missing. Code is available at https://github.com/Kebii/R2ET.