mlsanityData to 14 Oct 2026

SinIR: Efficient General Image Manipulation with Single Image Reconstruction

arXiv 2106.07140 ↗First posted 14 Jun 2021cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

We propose SinIR, an efficient reconstruction-based framework trained on a single natural image for general image manipulation, including super-resolution, editing, harmonization, paint-to-image, photo-realistic style transfer, and artistic style transfer. We train our model on a single image with cascaded multi-scale learning, where each network at each scale is responsible for image reconstruction. This reconstruction objective greatly reduces the complexity and running time of training, compared to the GAN objective. However, the reconstruction objective also exacerbates the output quality. Therefore, to solve this problem, we further utilize simple random pixel shuffling, which also gives control over manipulation, inspired by the Denoising Autoencoder. With quantitative evaluation, we show that SinIR has competitive performance on various image manipulation tasks. Moreover, with a much simpler training objective (i.e., reconstruction), SinIR is trained 33.5 times faster than SinGAN (for 500 X 500 images) that solves similar tasks. Our code is publicly available at github.com/YooJiHyeong/SinIR.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 37 of its 58 references that are in the map. First posted on arXiv 14 Jun 2021.

Builds on

37 of 58 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  6. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  7. 2017Improved Training of Wasserstein GANscited by 2,235
  8. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  9. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,552
  10. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
  11. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  12. 2017Deep Image Priorcited by 1,098
25 more
  1. 2017The Perception-Distortion Tradeoffcited by 587
  2. 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 364
  3. 2017Universal Style Transfer via Feature Transformscited by 354
  4. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  5. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 286
  6. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  7. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  8. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  9. 2016Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networkscited by 192
  10. 2019Image Processing Using Multi-Code GAN Priorcited by 185
  11. 2019Blind Super-Resolution Kernel Estimation using an Internal-GANcited by 175
  12. 2017Deep Photo Style Transfercited by 168
  13. 2019Photorealistic Style Transfer via Wavelet Transformscited by 166
  14. 2018"Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priorscited by 114
  15. 2019Neural Blind Deconvolution Using Deep Priorscited by 113
  16. 2018Non-Stationary Texture Synthesis by Adversarial Expansioncited by 67
  17. 2017Learning Texture Manifolds with the Periodic Spatial GANcited by 53
  18. 2018Semi-parametric Image Synthesiscited by 50
  19. 2016Texture Synthesis with Spatial Generative Adversarial Networkscited by 45
  20. 2016Stacked Generative Adversarial Networkscited by 44
  21. 2019Identity Crisis: Memorization and Generalization under Extreme Overparameterizationcited by 33
  22. 2019An Internal Learning Approach to Video Inpaintingcited by 33
  23. 2018Deep Painterly Harmonizationcited by 19
  24. 2019Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texturecited by 15
  25. 2019DCIL: Deep Contextual Internal Learning for Image Restoration and Image Retargetingcited by 10

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 7; 2023: 4; 2024: 1.

13 papers.

Sort:

By area:

  1. 2023Single Motion Diffusioncited by 65
  2. 2022SinDDM: A Single Image Denoising Diffusion Modelcited by 60
  3. 2022SinDiffusion: Learning a Diffusion Model from a Single Natural Imagecited by 42
  4. 2022MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translationcited by 22
  5. 2022PetsGAN: Rethinking Priors for Single Image Generationcited by 14
  6. 2023NLUT: Neural-based 3D Lookup Tables for Video Photorealistic Style Transfercited by 10
  7. 2021Deep Translation Prior: Test-time Training for Photorealistic Style Transfercited by 8
  8. 2022Controllable Style Transfer via Test-time Training of Implicit Neural Representationcited by 8
  9. 2023Aerial Diffusion: Text Guided Ground-to-Aerial View Translation from a Single Image using Diffusion Modelscited by 4
  10. 2023Learning in a Single Domain for Non-Stationary Multi-Texture Synthesiscited by 1
  11. 2024Towards Practical Single-shot Motion Synthesiscited by 1
  12. 2022Adaptive adversarial training method for improving multi-scale GAN based on generalization bound theorycited by 0
1 more
  1. 2022Vicious Classifiers: Assessing Inference-time Data Reconstruction Risk in Edge Computingcited by 0