# PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal

arXiv: https://arxiv.org/abs/2402.02374 · HTML: https://mlsanity.com/paper/2402.02374/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 39 of its 71 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 39)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [MaPLe: Multi-modal Prompt Learning](https://mlsanity.com/paper/2210.03117/) (2022)
- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [DiffIR: Efficient Diffusion Model for Image Restoration](https://mlsanity.com/paper/2303.09472/) (2023)
- [Multi-level Wavelet-CNN for Image Restoration](https://mlsanity.com/paper/1805.07071/) (2018)
- [EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis](https://mlsanity.com/paper/1612.07919/) (2016)
- [Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method](https://mlsanity.com/paper/2212.11548/) (2022)
- [Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2303.02861/) (2023)
- [Single Image Reflection Separation with Perceptual Losses](https://mlsanity.com/paper/1806.05376/) (2018)
- [Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation](https://mlsanity.com/paper/2212.04145/) (2022)
- [Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors](https://mlsanity.com/paper/2103.11298/) (2021)
- [HairCLIP: Design Your Hair by Text and Reference Image](https://mlsanity.com/paper/2112.05142/) (2021)
- [HyperPrompt: Prompt-based Task-Conditioning of Transformers](https://mlsanity.com/paper/2203.00759/) (2022)
- [A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing](https://mlsanity.com/paper/1708.03474/) (2017)
- [Adversarial Spatio-Temporal Learning for Video Deblurring](https://mlsanity.com/paper/1804.00533/) (2018)

## Built on by (most cited first, 14 of 14)

- [Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror Reflections](https://mlsanity.com/paper/2409.14677/) (2024)
- [Dereflection Any Image with Diffusion Priors and Diversified Data](https://mlsanity.com/paper/2503.17347/) (2025)
- [Flash-Split: 2D Reflection Removal with Flash Cues and Latent Diffusion Separation](https://mlsanity.com/paper/2501.00637/) (2024)
- [UCIP: A Universal Framework for Compressed Image Super-Resolution using Dynamic Prompt](https://mlsanity.com/paper/2407.13108/) (2024)
- [Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration](https://mlsanity.com/paper/2412.21042/) (2024)
- [UniSER: A Foundation Model for Unified Soft Effects Removal](https://mlsanity.com/paper/2511.14183/) (2025)
- [Reflection Removal through Efficient Adaptation of Diffusion Transformers](https://mlsanity.com/paper/2512.05000/) (2025)
- [Rectifying Latent Space for Generative Single-Image Reflection Removal](https://mlsanity.com/paper/2512.06358/) (2025)
- [TextDoctor: Unified Document Image Inpainting via Patch Pyramid Diffusion Models](https://mlsanity.com/paper/2503.04021/) (2025)
- [Diffusion-Based sRGB Real Noise Generation via Prompt-Driven Noise Representation Learning](https://mlsanity.com/paper/2603.04870/) (2026)
- [Reflection Removal Using Recurrent Polarization-to-Polarization Network](https://mlsanity.com/paper/2402.18178/) (2024)
- [DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration](https://mlsanity.com/paper/2411.10508/) (2024)
- [Reflection Separation from a Single Image via Joint Latent Diffusion](https://mlsanity.com/paper/2606.04107/) (2026)
- [From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection](https://mlsanity.com/paper/2608.11562/) (2026)

## Abstract (arXiv)

Existing single image reflection removal (SIRR) methods using deep learning tend to miss key low-frequency (LF) and high-frequency (HF) differences in images, affecting their effectiveness in removing reflections. To address this problem, this paper proposes a novel prompt-guided reflection removal (PromptRR) framework that uses frequency information as new visual prompts for better reflection performance. Specifically, the proposed framework decouples the reflection removal process into the prompt generation and subsequent prompt-guided restoration. For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts. Then, we adopt diffusion models (DMs) as prompt generators to generate the LF and HF prompts estimated by the pre-trained frequency prompt encoder. For the prompt-guided restoration, we integrate specially generated prompts into the PromptFormer network, employing a novel Transformer-based prompt block to effectively steer the model toward enhanced reflection removal. The results on commonly used benchmarks show that our method outperforms state-of-the-art approaches. The codes and models are available at https://github.com/TaoWangzj/PromptRR.