# TPC: Test-time Procrustes Calibration for Diffusion-based Human Image Animation

arXiv: https://arxiv.org/abs/2410.24037 · HTML: https://mlsanity.com/paper/2410.24037/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model](https://mlsanity.com/paper/2311.16498/) (2023)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [Thin-Plate Spline Motion Model for Image Animation](https://mlsanity.com/paper/2203.14367/) (2022)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion](https://mlsanity.com/paper/2304.06025/) (2023)
- [DisCo: Disentangled Control for Realistic Human Dance Generation](https://mlsanity.com/paper/2307.00040/) (2023)
- [Diffusion Probabilistic Model Made Slim](https://mlsanity.com/paper/2211.17106/) (2022)
- [FlexiEdit: Frequency-Aware Latent Refinement for Enhanced Non-Rigid Editing](https://mlsanity.com/paper/2407.17850/) (2024)
- [Selective Query-guided Debiasing for Video Corpus Moment Retrieval](https://mlsanity.com/paper/2210.08714/) (2022)

## Built on by (most cited first, 12 of 12)

- [Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance](https://mlsanity.com/paper/2502.06145/) (2025)
- [DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance](https://mlsanity.com/paper/2504.01724/) (2025)
- [ITA-MDT: Image-Timestep-Adaptive Masked Diffusion Transformer Framework for Image-Based Virtual Try-On](https://mlsanity.com/paper/2503.20418/) (2025)
- [TARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesis](https://mlsanity.com/paper/2504.05684/) (2025)
- [ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization](https://mlsanity.com/paper/2506.08712/) (2025)
- [FlowDrag: 3D-aware Drag-based Image Editing with Mesh-guided Deformation Vector Flow Fields](https://mlsanity.com/paper/2507.08285/) (2025)
- [Wan-Animate-2: Pushing the Application Boundaries of Character Animation](https://mlsanity.com/paper/2608.06009/) (2026)
- [Occlusion-robust Stylization for Drawing-based 3D Animation](https://mlsanity.com/paper/2508.00398/) (2025)
- [MotionWeaver: Holistic 4D-Anchored Framework for Multi-Humanoid Image Animation](https://mlsanity.com/paper/2602.13326/) (2026)
- [SPECSIA: Stylization Dataset for Novel-View Enhancement in Drawing-based 3D Animation](https://mlsanity.com/paper/2607.00525/) (2026)
- [InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360deg Image](https://mlsanity.com/paper/2607.03990/) (2026)
- [MultiAnimate: A Unified Framework for Controllable Multi-Character Animation](https://mlsanity.com/paper/2607.13415/) (2026)

## Abstract (arXiv)

Human image animation aims to generate a human motion video from the inputs of a reference human image and a target motion video. Current diffusion-based image animation systems exhibit high precision in transferring human identity into targeted motion, yet they still exhibit irregular quality in their outputs. Their optimal precision is achieved only when the physical compositions (i.e., scale and rotation) of the human shapes in the reference image and target pose frame are aligned. In the absence of such alignment, there is a noticeable decline in fidelity and consistency. Especially, in real-world environments, this compositional misalignment commonly occurs, posing significant challenges to the practical usage of current systems. To this end, we propose Test-time Procrustes Calibration (TPC), which enhances the robustness of diffusion-based image animation systems by maintaining optimal performance even when faced with compositional misalignment, effectively addressing real-world scenarios. The TPC provides a calibrated reference image for the diffusion model, enhancing its capability to understand the correspondence between human shapes in the reference and target images. Our method is simple and can be applied to any diffusion-based image animation system in a model-agnostic manner, improving the effectiveness at test time without additional training.