# AniDoc: Animation Creation Made Easier

arXiv: https://arxiv.org/abs/2412.14173 · HTML: https://mlsanity.com/paper/2412.14173/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 65 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 30 of 38)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction](https://mlsanity.com/paper/2310.20700/) (2023)
- [SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models](https://mlsanity.com/paper/2311.16933/) (2023)
- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)
- [Pixel Difference Networks for Efficient Edge Detection](https://mlsanity.com/paper/2108.07009/) (2021)
- [DragAnything: Motion Control for Anything using Entity Representation](https://mlsanity.com/paper/2403.07420/) (2024)
- [ID-Animator: Zero-Shot Identity-Preserving Human Video Generation](https://mlsanity.com/paper/2404.15275/) (2024)
- [LDMVFI: Video Frame Interpolation with Latent Diffusion Models](https://mlsanity.com/paper/2303.09508/) (2023)
- [Learning to generate line drawings that convey geometry and semantics](https://mlsanity.com/paper/2203.12691/) (2022)
- [Scribbler: Controlling Deep Image Synthesis with Sketch and Color](https://mlsanity.com/paper/1612.00835/) (2016)
- [Video Interpolation with Diffusion Models](https://mlsanity.com/paper/2404.01203/) (2024)
- [ToonCrafter: Generative Cartoon Interpolation](https://mlsanity.com/paper/2405.17933/) (2024)
- [Deep Animation Video Interpolation in the Wild](https://mlsanity.com/paper/2104.02495/) (2021)
- [Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss](https://mlsanity.com/paper/1908.05840/) (2019)
- [User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks](https://mlsanity.com/paper/1808.03240/) (2018)
- [LVCD: Reference-based Lineart Video Colorization with Diffusion Models](https://mlsanity.com/paper/2409.12960/) (2024)
- [Eliminating Gradient Conflict in Reference-based Line-Art Colorization](https://mlsanity.com/paper/2207.06095/) (2022)
- [X-Pose: Detecting Any Keypoints](https://mlsanity.com/paper/2310.08530/) (2023)
- [Improving the Perceptual Quality of 2D Animation Interpolation](https://mlsanity.com/paper/2111.12792/) (2021)

## Built on by (most cited first, 22 of 22)

- [Follow-Your-Creation: Empowering 4D Creation through Video Inpainting](https://mlsanity.com/paper/2506.04590/) (2025)
- [EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editing](https://mlsanity.com/paper/2503.10270/) (2025)
- [Generative AI for Cel-Animation: A Survey](https://mlsanity.com/paper/2501.06250/) (2025)
- [MangaNinja: Line Art Colorization with Precise Reference Following](https://mlsanity.com/paper/2501.08332/) (2025)
- [LayerAnimate: Layer-level Control for Animation](https://mlsanity.com/paper/2501.08295/) (2025)
- [Follow-Your-Color: Multi-Instance Sketch Colorization](https://mlsanity.com/paper/2503.16948/) (2025)
- [AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation](https://mlsanity.com/paper/2506.01061/) (2025)
- [ToonComposer: Streamlining Cartoon Production with Generative Post-Keyframing](https://mlsanity.com/paper/2508.10881/) (2025)
- [LongAnimation: Long Animation Generation with Dynamic Global-Local Memory](https://mlsanity.com/paper/2507.01945/) (2025)
- [DACoN: DINO for Anime Paint Bucket Colorization with Any Number of Reference Images](https://mlsanity.com/paper/2509.14685/) (2025)
- [SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning](https://mlsanity.com/paper/2603.10446/) (2026)
- [SketchColour: Channel Concat Guided DiT-based Sketch-to-Colour Pipeline for 2D Animation](https://mlsanity.com/paper/2507.01586/) (2025)
- [TimeColor: Flexible Reference Colorization via Temporal Concatenation](https://mlsanity.com/paper/2601.00296/) (2026)
- [Workflow-Aware Structured Layer Decomposition for Illustration Production](https://mlsanity.com/paper/2603.14925/) (2026)
- [InstanceAnimator: Multi-Instance Sketch Video Colorization](https://mlsanity.com/paper/2603.25357/) (2026)
- [OmniColor: A Unified Framework for Multi-modal Lineart Colorization](https://mlsanity.com/paper/2603.27531/) (2026)
- [MagicAnime: A Hierarchically Annotated, Multimodal and Multitasking Dataset with Benchmarks for Cartoon Animation Generation](https://mlsanity.com/paper/2507.20368/) (2025)
- [Region-Wise Correspondence Prediction between Manga Line Art Images](https://mlsanity.com/paper/2509.09501/) (2025)
- [Uni-Animator: Towards Unified Visual Colorization](https://mlsanity.com/paper/2602.23191/) (2026)
- [Towards High-resolution and Disentangled Reference-based Sketch Colorization](https://mlsanity.com/paper/2603.05971/) (2026)
- [SketchKeyAnime: Reference-anchored Sparse Key-Sketch Animation Synthesis](https://mlsanity.com/paper/2606.19958/) (2026)
- [PeCA: Palette Context Assisted Inference for Test-Time Paint-Bucket Colourisation on Animation Videos](https://mlsanity.com/paper/2608.00903/) (2026)

## Abstract (arXiv)

The production of 2D animation follows an industry-standard workflow, encompassing four essential stages: character design, keyframe animation, in-betweening, and coloring. Our research focuses on reducing the labor costs in the above process by harnessing the potential of increasingly powerful generative AI. Using video diffusion models as the foundation, AniDoc emerges as a video line art colorization tool, which automatically converts sketch sequences into colored animations following the reference character specification. Our model exploits correspondence matching as an explicit guidance, yielding strong robustness to the variations (e.g., posture) between the reference character and each line art frame. In addition, our model could even automate the in-betweening process, such that users can easily create a temporally consistent animation by simply providing a character image as well as the start and end sketches. Our code is available at: https://yihao-meng.github.io/AniDoc_demo.