# Why Cross-Skeleton Retargeting Is Non-Identifiable: Structural Limits of Generative Motion Models

arXiv: https://arxiv.org/abs/2609.37297 · HTML: https://mlsanity.com/paper/2609.37297/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 14 of 14)

- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling](https://mlsanity.com/paper/2106.01357/) (2021)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [Unpaired Image-to-Image Translation via Neural Schr"odinger Bridge](https://mlsanity.com/paper/2305.15086/) (2023)
- [Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking](https://mlsanity.com/paper/2510.02252/) (2025)
- [Single Motion Diffusion](https://mlsanity.com/paper/2302.05905/) (2023)
- [AnyTop: Character Animation Diffusion with Any Topology](https://mlsanity.com/paper/2502.17327/) (2025)
- [Motion2Motion: Cross-topology Motion Transfer with Sparse Correspondence](https://mlsanity.com/paper/2508.13139/) (2025)
- [ACE: Adversarial Correspondence Embedding for Cross Morphology Motion Retargeting from Human to Nonhuman Characters](https://mlsanity.com/paper/2305.14792/) (2023)
- [How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects](https://mlsanity.com/paper/2503.04257/) (2025)
- [MoReFlow: Motion Retargeting Learning through Unsupervised Flow Matching](https://mlsanity.com/paper/2509.25600/) (2025)
- [Topology-Agnostic Animal Motion Generation from Text Prompt](https://mlsanity.com/paper/2512.10352/) (2025)

## Abstract (arXiv)

Cross-skeleton motion generation trains generative models to carry action structure and motion intention from one body to another. Yet a target motion that shows the right action has two explanations that the training data cannot tell apart: the model transferred the source clip, or it recovered a typical motion for the requested action. We show that this ambiguity is structural rather than incidental: under standard generative objectives, the source-conditioned retargeting map is non-identifiable in sparse heterogeneous motion domains. Unpaired distribution matching yields gauge non-identifiability: the latent spaces of different skeletons can be transformed relative to one another without changing the training evidence, so different source-conditioned maps fit it equally well. Sparse paired supervision admits the complementary failure mode, conditional-mean degeneration: when clips are paired only by action, squared-error training converges to an average target motion that ignores the source clip. To make the missing evidence observable, we introduce Source-Instance Fidelity (SIF), a diagnostic that tests whether outputs differ from one another the way their source clips do, with the target skeleton and action held fixed. Under this diagnostic, methods that succeed at the standard action-level test on animal motion data often sit at the source-blind floor, while the methods that rise above it retain only a partial relational signal. Retargeting therefore needs objectives and evaluations that can identify the source-conditioned map it claims to learn. Project page: https://cross-skeleton-retargeting.netlify.app/.