# SCAIL-2: Unifying Controlled Character Animation with End-to-End In-Context Conditioning

arXiv: https://arxiv.org/abs/2606.10804 · HTML: https://mlsanity.com/paper/2606.10804/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Builds on (most cited first, 30 of 35)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel](https://mlsanity.com/paper/2304.11277/) (2023)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance](https://mlsanity.com/paper/2403.14781/) (2024)
- [Animate-X: Universal Character Image Animation with Enhanced Motion Representation](https://mlsanity.com/paper/2410.10306/) (2024)
- [Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning](https://mlsanity.com/paper/2506.05207/) (2025)
- [Wan-Animate: Unified Character Animation and Replacement with Holistic Replication](https://mlsanity.com/paper/2509.14055/) (2025)
- [SAM 3D Body: Robust Full-Body Human Mesh Recovery](https://mlsanity.com/paper/2602.15989/) (2026)
- [Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance](https://mlsanity.com/paper/2502.06145/) (2025)
- [Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2407.07532/) (2024)
- [Video-Bench: Human-Aligned Video Generation Benchmark](https://mlsanity.com/paper/2504.04907/) (2025)
- [HuMo: Human-Centric Video Generation via Collaborative Multi-Modal Conditioning](https://mlsanity.com/paper/2509.08519/) (2025)
- [UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer](https://mlsanity.com/paper/2504.11289/) (2025)
- [FastVMT: Eliminating Redundancy in Video Motion Transfer](https://mlsanity.com/paper/2602.05551/) (2026)
- [DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation](https://mlsanity.com/paper/2505.18078/) (2025)
- [X-UniMotion: Animating Human Images with Expressive, Unified and Identity-Agnostic Motion Latents](https://mlsanity.com/paper/2508.09383/) (2025)
- [SCAIL: Towards Studio-Grade Character Animation via In-Context Learning of 3D-Consistent Pose Representations](https://mlsanity.com/paper/2512.05905/) (2025)
- [SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation](https://mlsanity.com/paper/2511.19320/) (2025)
- [One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfer](https://mlsanity.com/paper/2511.22940/) (2025)
- [MoCha:End-to-End Video Character Replacement without Structural Guidance](https://mlsanity.com/paper/2601.08587/) (2026)
- [3D-Aware Implicit Motion Control for View-Adaptive Human Video Generation](https://mlsanity.com/paper/2602.03796/) (2026)

## Built on by (most cited first, 8 of 8)

- [Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation](https://mlsanity.com/paper/2609.11638/) (2026)
- [Wan-Animate-2: Pushing the Application Boundaries of Character Animation](https://mlsanity.com/paper/2608.06009/) (2026)
- [UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation](https://mlsanity.com/paper/2608.01944/) (2026)
- [Vorch-IR: Long-Form Unified Multimodal Identity Replacement Video Generation](https://mlsanity.com/paper/2608.05648/) (2026)
- [UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos](https://mlsanity.com/paper/2608.11752/) (2026)
- [EditaLive! Unified Character Video Editing for Live Streaming](https://mlsanity.com/paper/2608.27123/) (2026)
- [WeLike2Party! In-Context Motion Transfer for Multi-Human Image Animation](https://mlsanity.com/paper/2609.36937/) (2026)
- [PixReenact: Pixel-Conditioned Causal Video Diffusion for Streaming Head-Avatar Reenactment](https://mlsanity.com/paper/2610.05233/) (2026)

## Abstract (arXiv)

Controlled character animation aims to transfer motion from a driving sequence to a reference character. Prior works heavily rely on intermediate representations, such as pose skeletons for motion and masked backgrounds for environment, inevitably resulting in information loss. In this work, we present SCAIL-2, a framework that adopts an end-to-end driving paradigm by directly concatenating latent visual information to the model's input sequence. We enable end-to-end training through a data synthesis pipeline that produces MotionPair-60K, a curated dataset for several character animation subtasks. We unify the subtasks using decoupled conditions to accommodate different driving patterns, facilitated by In-Context Mask Conditioning and Mode-Specific RoPE, which provide soft guidance beyond textual instructions and visual information. To address synthetic discrepancy in detailed regions, we propose Bias-Aware DPO to construct preference items to mitigate the errors. Extensive experiments demonstrate that our method achieves state-of-the-art performance across various character animation tasks. Code, model weights, and a large subset of the dataset are available at: https://teal024.github.io/SCAIL-2/.