# SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution

arXiv: https://arxiv.org/abs/2201.04364 · HTML: https://mlsanity.com/paper/2201.04364/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 12 Jan 2022.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Model](https://mlsanity.com/paper/1904.00523/) (2019)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [Meta-SR: A Magnification-Arbitrary Network for Super-Resolution](https://mlsanity.com/paper/1903.00875/) (2019)
- [Zoom To Learn, Learn To Zoom](https://mlsanity.com/paper/1905.05169/) (2019)
- [Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2003.07018/) (2020)
- [Colorization Transformer](https://mlsanity.com/paper/2102.04432/) (2021)
- [Deep Exemplar-based Colorization](https://mlsanity.com/paper/1807.06587/) (2018)
- [Instance-aware Image Colorization](https://mlsanity.com/paper/2005.10825/) (2020)
- [Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/1906.09909/) (2019)
- [Deep Colorization](https://mlsanity.com/paper/1605.00075/) (2016)
- [ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution](https://mlsanity.com/paper/1907.09837/) (2019)
- [Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence](https://mlsanity.com/paper/2005.05207/) (2020)
- [TextureGAN: Controlling Deep Image Synthesis with Texture Patches](https://mlsanity.com/paper/1706.02823/) (2017)
- [Learning Diverse Image Colorization](https://mlsanity.com/paper/1612.01958/) (2016)
- [Fully Automatic Video Colorization with Self-Regularization and Diversity](https://mlsanity.com/paper/1908.01311/) (2019)

## Built on by (most cited first, 8 of 8)

- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [DiffUTE: Universal Text Editing Diffusion Model](https://mlsanity.com/paper/2305.10825/) (2023)
- [ColorFlow: Retrieval-Augmented Image Sequence Colorization](https://mlsanity.com/paper/2412.11815/) (2024)
- [Learning Global-aware Kernel for Image Harmonization](https://mlsanity.com/paper/2305.11676/) (2023)
- [Exploring Real&Synthetic Dataset and Linear Attention in Image Restoration](https://mlsanity.com/paper/2412.03814/) (2024)
- [From Fragment to One Piece: A Survey on AI-Driven Graphic Design](https://mlsanity.com/paper/2503.18641/) (2025)
- [Computer-aided Colorization State-of-the-science: A Survey](https://mlsanity.com/paper/2410.02288/) (2024)
- [Leveraging the Powerful Attention of a Pre-trained Diffusion Model for Exemplar-based Image Colorization](https://mlsanity.com/paper/2505.15812/) (2025)

## Abstract (arXiv)

In the practical application of restoring low-resolution gray-scale images, we generally need to run three separate processes of image colorization, super-resolution, and dows-sampling operation for the target device. However, this pipeline is redundant and inefficient for the independent processes, and some inner features could have been shared. Therefore, we present an efficient paradigm to perform Simultaneously Image Colorization and Super-resolution (SCS) and propose an end-to-end SCSNet to achieve this goal. The proposed method consists of two parts: colorization branch for learning color information that employs the proposed plug-and-play Pyramid Valve Cross Attention (PVCAttn) module to aggregate feature maps between source and reference images; and super-resolution branch for integrating color and texture information to predict target images, which uses the designed Continuous Pixel Mapping (CPM) module to predict high-resolution images at continuous magnification. Furthermore, our SCSNet supports both automatic and referential modes that is more flexible for practical application. Abundant experiments demonstrate the superiority of our method for generating authentic images over state-of-the-art methods, e.g., averagely decreasing FID by 1.8downarrow and 5.1 downarrow compared with current best scores for automatic and referential modes, respectively, while owning fewer parameters (more than ×2downarrow) and faster running speed (more than ×3uparrow).