# Pose-aware Attention Network for Flexible Motion Retargeting by Body Part

arXiv: https://arxiv.org/abs/2306.08006 · HTML: https://mlsanity.com/paper/2306.08006/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 57 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video](https://mlsanity.com/paper/2203.00859/) (2022)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [Contact and Human Dynamics from Monocular Video](https://mlsanity.com/paper/2007.11678/) (2020)
- [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018)
- [Motion Puzzle: Arbitrary Motion Style Transfer by Body Part](https://mlsanity.com/paper/2202.05274/) (2022)
- [Spatio-Temporal Gating-Adjacency GCN for Human Motion Prediction](https://mlsanity.com/paper/2203.01474/) (2022)
- [Learning Character-Agnostic Motion for Motion Retargeting in 2D](https://mlsanity.com/paper/1905.01680/) (2019)
- [Contact-Aware Retargeting of Skinned Motion](https://mlsanity.com/paper/2109.07431/) (2021)
- [Skeleton-free Pose Transfer for Stylized 3D Characters](https://mlsanity.com/paper/2208.00790/) (2022)
- [Neural3Points: Learning to Generate Physically Realistic Full-body Motion for Virtual Reality Users](https://mlsanity.com/paper/2209.05753/) (2022)
- [Flow Guided Transformable Bottleneck Networks for Motion Retargeting](https://mlsanity.com/paper/2106.07771/) (2021)
- [Human Motion Control of Quadrupedal Robots using Deep Reinforcement Learning](https://mlsanity.com/paper/2204.13336/) (2022)

## Built on by (most cited first, 10 of 10)

- [AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism](https://mlsanity.com/paper/2309.00796/) (2023)
- [STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargeting](https://mlsanity.com/paper/2406.04629/) (2024)
- [Diffusion-based Human Motion Style Transfer with Semantic Guidance](https://mlsanity.com/paper/2405.06646/) (2024)
- [Semantics-aware Motion Retargeting with Vision-Language Models](https://mlsanity.com/paper/2312.01964/) (2023)
- [HMVLM: Human Motion-Vision-Lanuage Model via MoE LoRA](https://mlsanity.com/paper/2511.01463/) (2025)
- [PALUM: Part-based Attention Learning for Unified Motion Retargeting](https://mlsanity.com/paper/2601.07272/) (2026)
- [ReActor: Reinforcement Learning for Physics-Aware Motion Retargeting](https://mlsanity.com/paper/2605.06593/) (2026)
- [AnyAct: Towards Human Reenactment of Character Motion From Video](https://mlsanity.com/paper/2605.15497/) (2026)
- [ReFM: Semantic-Aware Refinement Flow Model for Motion Retargeting](https://mlsanity.com/paper/2609.32068/) (2026)
- [What Makes a Latent Space Shared? Direct and Recoverable Compatibility in Motion VAEs](https://mlsanity.com/paper/2610.17376/) (2026)

## Abstract (arXiv)

Motion retargeting is a fundamental problem in computer graphics and computer vision. Existing approaches usually have many strict requirements, such as the source-target skeletons needing to have the same number of joints or share the same topology. To tackle this problem, we note that skeletons with different structure may have some common body parts despite the differences in joint numbers. Following this observation, we propose a novel, flexible motion retargeting framework. The key idea of our method is to regard the body part as the basic retargeting unit rather than directly retargeting the whole body motion. To enhance the spatial modeling capability of the motion encoder, we introduce a pose-aware attention network (PAN) in the motion encoding phase. The PAN is pose-aware since it can dynamically predict the joint weights within each body part based on the input pose, and then construct a shared latent space for each body part by feature pooling. Extensive experiments show that our approach can generate better motion retargeting results both qualitatively and quantitatively than state-of-the-art methods. Moreover, we also show that our framework can generate reasonable results even for a more challenging retargeting scenario, like retargeting between bipedal and quadrupedal skeletons because of the body part retargeting strategy and PAN. Our code is publicly available.