# Learning Data-Driven Vector-Quantized Degradation Model for Animation Video Super-Resolution

arXiv: https://arxiv.org/abs/2303.09826 · HTML: https://mlsanity.com/paper/2303.09826/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 53 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 30 of 44)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Learning a Single Convolutional Super-Resolution Network for Multiple Degradations](https://mlsanity.com/paper/1712.06116/) (2017)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [Blind Super-Resolution With Iterative Kernel Correction](https://mlsanity.com/paper/1904.03377/) (2019)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Investigating Tradeoffs in Real-World Video Super-Resolution](https://mlsanity.com/paper/2111.12704/) (2021)
- [Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks](https://mlsanity.com/paper/1809.00437/) (2018)
- [Video Super-Resolution with Recurrent Structure-Detail Network](https://mlsanity.com/paper/2008.00455/) (2020)
- [VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder](https://mlsanity.com/paper/2205.06803/) (2022)
- [Unsupervised Real-world Image Super Resolution via Domain-distance Aware Training](https://mlsanity.com/paper/2004.01178/) (2020)
- [Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/1906.09909/) (2019)
- [Efficient Video Super-Resolution through Recurrent Latent Space Propagation](https://mlsanity.com/paper/1909.08080/) (2019)
- [Unpaired Image Super-Resolution using Pseudo-Supervision](https://mlsanity.com/paper/2002.11397/) (2020)
- [Deep Animation Video Interpolation in the Wild](https://mlsanity.com/paper/2104.02495/) (2021)
- [Learning Trajectory-Aware Transformer for Video Super-Resolution](https://mlsanity.com/paper/2204.04216/) (2022)

## Built on by (most cited first, 3 of 3)

- [APISR: Anime Production Inspired Real-World Anime Super-Resolution](https://mlsanity.com/paper/2403.01598/) (2024)
- [Learning Profitable NFT Image Diffusions via Multiple Visual-Policy Guided Reinforcement Learning](https://mlsanity.com/paper/2306.11731/) (2023)
- [RustNeRF: Robust Neural Radiance Field with Low-Quality Images](https://mlsanity.com/paper/2401.03257/) (2024)

## Abstract (arXiv)

Existing real-world video super-resolution (VSR) methods focus on designing a general degradation pipeline for open-domain videos while ignoring data intrinsic characteristics which strongly limit their performance when applying to some specific domains (eg., animation videos). In this paper, we thoroughly explore the characteristics of animation videos and leverage the rich priors in real-world animation data for a more practical animation VSR model. In particular, we propose a multi-scale Vector-Quantized Degradation model for animation video Super-Resolution (VQD-SR) to decompose the local details from global structures and transfer the degradation priors in real-world animation videos to a learned vector-quantized codebook for degradation modeling. A rich-content Real Animation Low-quality (RAL) video dataset is collected for extracting the priors. We further propose a data enhancement strategy for high-resolution (HR) training videos based on our observation that existing HR videos are mostly collected from the Web which contains conspicuous compression artifacts. The proposed strategy is valid to lift the upper bound of animation VSR performance, regardless of the specific VSR model. Experimental results demonstrate the superiority of the proposed VQD-SR over state-of-the-art methods, through extensive quantitative and qualitative evaluations of the latest animation video super-resolution benchmark. The code and pre-trained models can be downloaded at https://github.com/researchmm/VQD-SR.