# Timestep-Aware Diffusion Model for Extreme Image Rescaling arXiv: https://arxiv.org/abs/2408.09151 ยท HTML: https://mlsanity.com/paper/2408.09151/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. - Builds on 33 of its 48 references that are in the map. - First posted on arXiv 17 Aug 2024. ## Placed in - Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/) ## Builds on (most cited first, 30 of 33) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019) - [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020) - [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017) - [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021) - [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022) - [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018) - [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021) - [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022) - [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020) - [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023) - [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021) - [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019) - [ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting](https://mlsanity.com/paper/2307.12348/) (2023) - [SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution](https://mlsanity.com/paper/2311.16518/) (2023) - [Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization](https://mlsanity.com/paper/2308.14469/) (2023) - [One-Step Effective Diffusion Network for Real-World Image Super-Resolution](https://mlsanity.com/paper/2406.08177/) (2024) - [SinSR: Diffusion-Based Image Super-Resolution in a Single Step](https://mlsanity.com/paper/2311.14760/) (2023) - [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023) - [Image and Video Compression with Neural Networks: A Review](https://mlsanity.com/paper/1904.03567/) (2019) - [Distilling Diffusion Models into Conditional GANs](https://mlsanity.com/paper/2405.05967/) (2024) - [Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling](https://mlsanity.com/paper/2108.05301/) (2021) - [Learned Image Downscaling for Upscaling using Content Adaptive Resampler](https://mlsanity.com/paper/1907.12904/) (2019) - [Lossy Image Compression with Foundation Diffusion Models](https://mlsanity.com/paper/2404.08580/) (2024) - [Towards Bidirectional Arbitrary Image Rescaling: Joint Optimization and Cycle Idempotence](https://mlsanity.com/paper/2203.00911/) (2022) - [Invertible Rescaling Network and Its Extensions](https://mlsanity.com/paper/2210.04188/) (2022) - [Not All Steps are Created Equal: Selective Diffusion Distillation for Image Manipulation](https://mlsanity.com/paper/2307.08448/) (2023) - [Self-Asymmetric Invertible Network for Compression-Aware Image Rescaling](https://mlsanity.com/paper/2303.02353/) (2023) - [Approaching the Limit of Image Rescaling via Flow Guidance](https://mlsanity.com/paper/2111.05133/) (2021) ## Built on by (most cited first, 5 of 5) - [Enhancing Low-resolution Image Representation Through Normalizing Flows](https://mlsanity.com/paper/2601.06834/) (2026) - [Joint Degradation-Aware Arbitrary-Scale Super-Resolution for Variable-Rate Extreme Image Compression](https://mlsanity.com/paper/2603.17408/) (2026) - [Faithful Extreme Image Rescaling with Learnable Reversible Transformation and Semantic Priors](https://mlsanity.com/paper/2605.00605/) (2026) - [SlimDiffSR: Toward Lightweight and Efficient Remote Sensing Image Super-Resolution via Diffusion Model Distillation](https://mlsanity.com/paper/2605.02198/) (2026) - [TVRN: Invertible Neural Networks for Compression-Aware Temporal Video Rescaling](https://mlsanity.com/paper/2605.15579/) (2026) ## Abstract (arXiv) Image rescaling aims to learn the optimal low-resolution (LR) image that can be accurately reconstructed to its original high-resolution (HR) counterpart, providing an efficient image processing and storage method for ultra-high definition media. However, extreme downscaling factors pose significant challenges to the upscaling process due to its highly ill-posed nature, causing existing image rescaling methods to struggle in generating semantically correct structures and perceptual friendly textures. In this work, we propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling, which performs rescaling operations in the latent space of a pre-trained autoencoder and effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model. Specifically, TADM adopts a pseudo-invertible module to establish the bidirectional mapping between the latent features of the HR image and the target-sized LR image. Then, the rescaled latent features are enhanced by a pre-trained diffusion model to generate more faithful details. Considering the spatially non-uniform degradation caused by the rescaling operation, we propose a novel time-step alignment strategy, which can adaptively allocate the generative capacity of the diffusion model based on the quality of the reconstructed latent features. Extensive experiments demonstrate the superiority of TADM over previous methods in both quantitative and qualitative evaluations.