# AnimeColor: Reference-based Animation Colorization with Diffusion Transformers

arXiv: https://arxiv.org/abs/2507.20158 · HTML: https://mlsanity.com/paper/2507.20158/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 27 Jul 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning](https://mlsanity.com/paper/2404.16994/) (2024)
- [AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One](https://mlsanity.com/paper/2312.06709/) (2023)
- [In-Context LoRA for Diffusion Transformers](https://mlsanity.com/paper/2410.23775/) (2024)
- [Learning to generate line drawings that convey geometry and semantics](https://mlsanity.com/paper/2203.12691/) (2022)
- [ToonCrafter: Generative Cartoon Interpolation](https://mlsanity.com/paper/2405.17933/) (2024)
- [Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss](https://mlsanity.com/paper/1908.05840/) (2019)
- [LVCD: Reference-based Lineart Video Colorization with Diffusion Models](https://mlsanity.com/paper/2409.12960/) (2024)
- [Automatic Temporally Coherent Video Colorization](https://mlsanity.com/paper/1904.09527/) (2019)
- [Learning Inclusion Matching for Animation Paint Bucket Colorization](https://mlsanity.com/paper/2403.18342/) (2024)
- [ColorizeDiffusion: Adjustable Sketch Colorization with Reference Image and Text](https://mlsanity.com/paper/2401.01456/) (2024)

## Built on by (most cited first, 5 of 5)

- [TimeColor: Flexible Reference Colorization via Temporal Concatenation](https://mlsanity.com/paper/2601.00296/) (2026)
- [InstanceAnimator: Multi-Instance Sketch Video Colorization](https://mlsanity.com/paper/2603.25357/) (2026)
- [OmniColor: A Unified Framework for Multi-modal Lineart Colorization](https://mlsanity.com/paper/2603.27531/) (2026)
- [Uni-Animator: Towards Unified Visual Colorization](https://mlsanity.com/paper/2602.23191/) (2026)
- [PeCA: Palette Context Assisted Inference for Test-Time Paint-Bucket Colourisation on Animation Videos](https://mlsanity.com/paper/2608.00903/) (2026)

## Abstract (arXiv)

Animation colorization plays a vital role in animation production, yet existing methods struggle to achieve color accuracy and temporal consistency. To address these challenges, we propose AnimeColor, a novel reference-based animation colorization framework leveraging Diffusion Transformers (DiT). Our approach integrates sketch sequences into a DiT-based video diffusion model, enabling sketch-controlled animation generation. We introduce two key components: a High-level Color Extractor (HCE) to capture semantic color information and a Low-level Color Guider (LCG) to extract fine-grained color details from reference images. These components work synergistically to guide the video diffusion process. Additionally, we employ a multi-stage training strategy to maximize the utilization of reference image color information. Extensive experiments demonstrate that AnimeColor outperforms existing methods in color accuracy, sketch alignment, temporal consistency, and visual quality. Our framework not only advances the state of the art in animation colorization but also provides a practical solution for industrial applications. The code will be made publicly available at hrefhttps://github.com/IamCreateAI/AnimeColorhttps://github.com/IamCreateAI/AnimeColor.