# PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models

arXiv: https://arxiv.org/abs/2312.13964 · HTML: https://mlsanity.com/paper/2312.13964/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 45 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval](https://mlsanity.com/paper/2104.00650/) (2021)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [Structure and Content-Guided Video Synthesis with Diffusion Models](https://mlsanity.com/paper/2302.03011/) (2023)
- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2303.08320/) (2023)
- [Flexible Diffusion Modeling of Long Videos](https://mlsanity.com/paper/2205.11495/) (2022)
- [Diffusion Probabilistic Modeling for Video Generation](https://mlsanity.com/paper/2203.09481/) (2022)
- [A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting](https://mlsanity.com/paper/2312.03594/) (2023)
- [DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation](https://mlsanity.com/paper/2301.03786/) (2023)

## Built on by (most cited first, 30 of 30)

- [DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmark](https://mlsanity.com/paper/2405.19707/) (2024)
- [RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control](https://mlsanity.com/paper/2502.10059/) (2025)
- [Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts](https://mlsanity.com/paper/2403.08268/) (2024)
- [Vivid-ZOO: Multi-View Video Generation with Diffusion Model](https://mlsanity.com/paper/2406.08659/) (2024)
- [I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strength](https://mlsanity.com/paper/2411.06525/) (2024)
- [StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration](https://mlsanity.com/paper/2411.04925/) (2024)
- [Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model](https://mlsanity.com/paper/2406.15735/) (2024)
- [I2VControl: Disentangled and Unified Video Motion Synthesis Control](https://mlsanity.com/paper/2411.17765/) (2024)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [UniCtrl: Improving the Spatiotemporal Consistency of Text-to-Video Diffusion Models via Training-Free Unified Attention Control](https://mlsanity.com/paper/2403.02332/) (2024)
- [Cinemo: Consistent and Controllable Image Animation with Motion Diffusion Models](https://mlsanity.com/paper/2407.15642/) (2024)
- [FaceShot: Bring Any Character into Life](https://mlsanity.com/paper/2503.00740/) (2025)
- [AtomoVideo: High Fidelity Image-to-Video Generation](https://mlsanity.com/paper/2403.01800/) (2024)
- [Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think](https://mlsanity.com/paper/2503.00948/) (2025)
- [MotiF: Making Text Count in Image Animation with Motion Focal Loss](https://mlsanity.com/paper/2412.16153/) (2024)
- [Any2Caption:Interpreting Any Condition to Caption for Controllable Video Generation](https://mlsanity.com/paper/2503.24379/) (2025)
- [VidSketch: Hand-drawn Sketch-Driven Video Generation with Diffusion Control](https://mlsanity.com/paper/2502.01101/) (2025)
- [GenHSI: Controllable Generation of Human-Scene Interaction Videos](https://mlsanity.com/paper/2506.19840/) (2025)
- [HANDI: Hand-Centric Text-and-Image Conditioned Video Generation](https://mlsanity.com/paper/2412.04189/) (2024)
- [Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicality](https://mlsanity.com/paper/2406.08845/) (2024)
- [InteractiveVideo: User-Centric Controllable Video Generation with Synergistic Multimodal Instructions](https://mlsanity.com/paper/2402.03040/) (2024)
- [Improving Multi-Subject Consistency in Open-Domain Image Generation with Isolation and Reposition Attention](https://mlsanity.com/paper/2411.19261/) (2024)
- [Consistent and Controllable Image Animation with Motion Linear Diffusion Transformers](https://mlsanity.com/paper/2508.07246/) (2025)
- [Image-to-Video Diffusion: From Foundations to Open Frontiers](https://mlsanity.com/paper/2605.17248/) (2026)
- [MIMAFace: Face Animation via Motion-Identity Modulated Appearance Feature Learning](https://mlsanity.com/paper/2409.15179/) (2024)
- [Learning to Animate Images from A Few Videos to Portray Delicate Human Actions](https://mlsanity.com/paper/2503.00276/) (2025)
- [Zero-Shot Personalized Camera Motion Control for Image-to-Video Synthesis](https://mlsanity.com/paper/2504.09472/) (2025)
- [Unified Text-Image-to-Video Generation: A Training-Free Approach to Flexible Visual Conditioning](https://mlsanity.com/paper/2505.20629/) (2025)
- [Few-Shot-Based Modular Image-to-Video Adapter for Diffusion Models](https://mlsanity.com/paper/2512.20000/) (2025)
- [PhysLayer: Language-Guided Layered Animation with Depth-Aware Physics](https://mlsanity.com/paper/2604.23574/) (2026)

## Abstract (arXiv)

Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles. While promising, adding realistic motions into these personalized images by text poses significant challenges in preserving distinct styles, high-fidelity details, and achieving motion controllability by text. In this paper, we present PIA, a Personalized Image Animator that excels in aligning with condition images, achieving motion controllability by text, and the compatibility with various personalized T2I models without specific tuning. To achieve these goals, PIA builds upon a base T2I model with well-trained temporal alignment layers, allowing for the seamless transformation of any personalized T2I model into an image animation model. A key component of PIA is the introduction of the condition module, which utilizes the condition frame and inter-frame affinity as input to transfer appearance information guided by the affinity hint for individual frame synthesis in the latent space. This design mitigates the challenges of appearance-related image alignment within and allows for a stronger focus on aligning with motion-related guidance.