# Identity-Preserving Image-to-Video Generation via Reward-Guided Optimization

arXiv: https://arxiv.org/abs/2510.14255 · HTML: https://mlsanity.com/paper/2510.14255/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 34 of its 36 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Diffusion Model Alignment Using Direct Preference Optimization](https://mlsanity.com/paper/2311.12908/) (2023)
- [Flow-GRPO: Training Flow Matching Models via Online RL](https://mlsanity.com/paper/2505.05470/) (2025)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Directly Fine-Tuning Diffusion Models on Differentiable Rewards](https://mlsanity.com/paper/2309.17400/) (2023)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [DanceGRPO: Unleashing GRPO on Visual Generation](https://mlsanity.com/paper/2505.07818/) (2025)
- [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020)
- [Aligning Text-to-Image Diffusion Models with Reward Backpropagation](https://mlsanity.com/paper/2310.03739/) (2023)
- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025)
- [Identity-Preserving Text-to-Video Generation by Frequency Decomposition](https://mlsanity.com/paper/2411.17440/) (2024)
- [Phantom: Subject-consistent video generation via cross-modal alignment](https://mlsanity.com/paper/2502.11079/) (2025)
- [ID-Animator: Zero-Shot Identity-Preserving Human Video Generation](https://mlsanity.com/paper/2404.15275/) (2024)
- [Video Diffusion Alignment via Reward Gradients](https://mlsanity.com/paper/2407.08737/) (2024)
- [Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library](https://mlsanity.com/paper/2506.06122/) (2025)
- [Sample and Computation Redistribution for Efficient Face Detection](https://mlsanity.com/paper/2105.04714/) (2021)
- [PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation](https://mlsanity.com/paper/2411.17048/) (2024)
- [Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation](https://mlsanity.com/paper/2508.07901/) (2025)
- [FantasyID: Face Knowledge Enhanced ID-Preserving Video Generation](https://mlsanity.com/paper/2502.13995/) (2025)
- [MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customization](https://mlsanity.com/paper/2503.12689/) (2025)

## Built on by (most cited first, 8 of 8)

- [Video-as-Answer: Predict and Generate Next Video Event with Joint-GRPO](https://mlsanity.com/paper/2511.16669/) (2025)
- [Scaling Zero-Shot Reference-to-Video Generation](https://mlsanity.com/paper/2512.06905/) (2025)
- [The World is Your Canvas: Painting Promptable Events with Reference Images, Trajectories, and Text](https://mlsanity.com/paper/2512.16924/) (2025)
- [DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning](https://mlsanity.com/paper/2603.12257/) (2026)
- [Advances in GRPO for Generation Models: A Survey](https://mlsanity.com/paper/2603.06623/) (2026)
- [ID-V2V: Identity-Preserving Video Restylization](https://mlsanity.com/paper/2607.22830/) (2026)
- [Video Generation Models: A Survey of Post-Training and Alignment](https://mlsanity.com/paper/2610.00812/) (2026)
- [Towards Subject Consistency over Dynamic Subject Sets in Video Generation](https://mlsanity.com/paper/2610.01052/) (2026)

## Abstract (arXiv)

Recent advances in image-to-video (I2V) generation have achieved remarkable progress in synthesizing high-quality, temporally coherent videos from static images. Among all the applications of I2V, human-centric video generation includes a large portion. However, existing I2V models encounter difficulties in maintaining identity consistency between the input human image and the generated video, especially when the person in the video exhibits significant expression changes and movements. This issue becomes critical when the human face occupies merely a small fraction of the image. Since humans are highly sensitive to identity variations, this poses a critical yet under-explored challenge in I2V generation. In this paper, we propose Identity-Preserving Reward-guided Optimization (IPRO), a novel video diffusion framework based on reinforcement learning to enhance identity preservation. Instead of introducing auxiliary modules or altering model architectures, our approach introduces a direct and effective tuning algorithm that optimizes diffusion models using a face identity scorer. To improve performance and accelerate convergence, our method backpropagates the reward signal through the last steps of the sampling chain, enabling richer gradient feedback. We also propose a novel facial scoring mechanism that treats faces in ground-truth videos as facial feature pools, providing multi-angle facial information to enhance generalization. A KL-divergence regularization is further incorporated to stabilize training and prevent overfitting to the reward signal. Extensive experiments on Wan 2.2 I2V model and our in-house I2V model demonstrate the effectiveness of our method. Our project and code are available at https://ipro-alimama.github.io/.