Fill2SR: Repurposing Inpainting Diffusion Transformers for Real-World Super-Resolution
On the map: Diffusion models → Inverse problems and image restoration
Abstract (arXiv)
Recent real-world image super-resolution (SR) methods often adapt text-to-image (T2I) backbones with ControlNet-style branches or spatial conditioning tokens, which increases memory and computes with resolution and often constrains training to a fixed scale. We propose Fill2SR, which repurposes a masked-inpainting Diffusion Transformer for SR without extra spatial branches. Our Inpainting-Interface Evidence Adapter (IIEA) writes the low-quality (LQ) observation into the native masked-image slot under a full-image mask, turning inpainting into a reverse-degradation conditional rectified flow trained with LoRA-only tuning. We further introduce RCDT, an offline pipeline that distills degradation descriptors from unpaired real images and transfers them onto clean targets using frozen open-source models. Fill2SR supports mixed-resolution training up to QHD and yields stable performance across 512/1024/2048 outputs. On synthetic benchmarks, our base model with IIEA achieves the best LPIPS on DIV2K and LSDIR; adding RCDT trades a small LPIPS drop for consistently stronger no-reference quality on RealLQ250 and RealPhoto60. Fill2SR remains memory-predictable, running 15362 inference on a single 32GB GPU and extending to multi-megapixel outputs via tiled restoration.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 41 of its 47 references that are in the map. First posted on arXiv 27 Sep 2026.
Builds on
41 of 47 references.
Closest ancestors · 8
Same direction, newest first
- 2025DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution30 Mar
- 2024FaithDiff: Unleashing Diffusion Priors for Faithful Image Super-resolution27 Nov
- 2024DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation24 Oct
- 2024One-Step Effective Diffusion Network for Real-World Image Super-Resolution12 Jun
- 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution27 Nov
- 2023SinSR: Diffusion-Based Image Super-Resolution in a Single Step23 Nov
- 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting23 Jul
- 2023Exploiting Diffusion Prior for Real-World Image Super-Resolution11 May
Other references · 33
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2023Visual Instruction Tuningcited by 7,869
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
21 more
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
- 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
- 2021Image Super-Resolution via Iterative Refinementcited by 1,511
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
- 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
- 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
- 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
- 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 866
- 2017The Perception-Distortion Tradeoffcited by 587
- 2021Designing a Practical Degradation Model for Deep Blind Image Super-Resolutioncited by 529
- 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 522
- 2019KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessmentcited by 355
- 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 286
- 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 278
- 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 169
- 2022Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolutioncited by 96
- 2025Acquire and then Adapt: Squeezing out Text-to-Image Model for Image Restorationcited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Degradation-Aware and Structure-Preserving Diffusion for Real-World Image Super-Resolution
- 2026TEASR: Training-Efficient Any-Step Diffusion Transformer for Real-World Image Super-Resolution
- 2026Q-DiT4SR: Exploration of Detail-Preserving Diffusion Transformer Quantization for Real-World Image Super-Resolution
- 2024Zoomed In, Diffused Out: Towards Local Degradation-Aware Multi-Diffusion for Extreme Image Super-Resolution
- 2026When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution
- 2026In-Token Learning for High-Fidelity Image Restoration via Diffusion Transformers
- 2024Effective Diffusion Transformer Architecture for Image Super-Resolution
- 2024TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution