# Approaching the Limit of Image Rescaling via Flow Guidance

arXiv: https://arxiv.org/abs/2111.05133 · HTML: https://mlsanity.com/paper/2111.05133/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 9 Nov 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [Unfolding the Alternating Optimization for Blind Super Resolution](https://mlsanity.com/paper/2010.02631/) (2020)
- [Invertible Image Rescaling](https://mlsanity.com/paper/2005.05650/) (2020)
- [Learned Image Downscaling for Upscaling using Content Adaptive Resampler](https://mlsanity.com/paper/1907.12904/) (2019)
- [Learning Spatial Attention for Face Super-Resolution](https://mlsanity.com/paper/2012.01211/) (2020)
- [FloWaveNet : A Generative Flow for Raw Audio](https://mlsanity.com/paper/1811.02155/) (2018)
- [C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds](https://mlsanity.com/paper/1912.07009/) (2019)
- [Conditional Adversarial Generative Flow for Controllable Image Synthesis](https://mlsanity.com/paper/1904.01782/) (2019)
- [Deep Feature Consistent Deep Image Transformations: Downscaling, Decolorization and HDR Tone Mapping](https://mlsanity.com/paper/1707.09482/) (2017)

## Built on by (most cited first, 8 of 8)

- [MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction](https://mlsanity.com/paper/2204.07908/) (2022)
- [Learning the Degradation Distribution for Blind Image Super-Resolution](https://mlsanity.com/paper/2203.04962/) (2022)
- [Downscaled Representation Matters: Improving Image Rescaling with Collaborative Downscaled Images](https://mlsanity.com/paper/2211.10643/) (2022)
- [Timestep-Aware Diffusion Model for Extreme Image Rescaling](https://mlsanity.com/paper/2408.09151/) (2024)
- [Enhancing Image Rescaling using Dual Latent Variables in Invertible Neural Network](https://mlsanity.com/paper/2207.11844/) (2022)
- [Effective Invertible Arbitrary Image Rescaling](https://mlsanity.com/paper/2209.13055/) (2022)
- [Noise-Started One-Step Real-World Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement](https://mlsanity.com/paper/2605.09328/) (2026)
- [Raising The Limit Of Image Rescaling Using Auxiliary Encoding](https://mlsanity.com/paper/2303.06747/) (2023)

## Abstract (arXiv)

Image downscaling and upscaling are two basic rescaling operations. Once the image is downscaled, it is difficult to be reconstructed via upscaling due to the loss of information. To make these two processes more compatible and improve the reconstruction performance, some efforts model them as a joint encoding-decoding task, with the constraint that the downscaled (i.e. encoded) low-resolution (LR) image must preserve the original visual appearance. To implement this constraint, most methods guide the downscaling module by supervising it with the bicubically downscaled LR version of the original high-resolution (HR) image. However, this bicubic LR guidance may be suboptimal for the subsequent upscaling (i.e. decoding) and restrict the final reconstruction performance. In this paper, instead of directly applying the LR guidance, we propose an additional invertible flow guidance module (FGM), which can transform the downscaled representation to the visually plausible image during downscaling and transform it back during upscaling. Benefiting from the invertibility of FGM, the downscaled representation could get rid of the LR guidance and would not disturb the downscaling-upscaling process. It allows us to remove the restrictions on the downscaling module and optimize the downscaling and upscaling modules in an end-to-end manner. In this way, these two modules could cooperate to maximize the HR reconstruction performance. Extensive experiments demonstrate that the proposed method can achieve state-of-the-art (SotA) performance on both downscaled and reconstructed images.