Data to 7 Oct 2026

Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks

arXiv 2403.00644 ↗First posted 1 Mar 2024cs.CVcited by 47

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models trained on large-scale datasets have achieved remarkable progress in image synthesis. However, due to the randomness in the diffusion process, they often struggle with handling diverse low-level tasks that require details preservation. To overcome this limitation, we present a new Diff-Plugin framework to enable a single pre-trained diffusion model to generate high-fidelity results across a variety of low-level tasks. Specifically, we first propose a lightweight Task-Plugin module with a dual branch design to provide task-specific priors, guiding the diffusion process in preserving image content. We then propose a Plugin-Selector that can automatically select different Task-Plugins based on the text instruction, allowing users to edit images by indicating multiple low-level tasks with natural language. We conduct extensive experiments on 8 low-level vision tasks. The results demonstrate the superiority of Diff-Plugin over existing methods, particularly in real-world scenarios. Our ablations further validate that Diff-Plugin is stable, schedulable, and supports robust training across different dataset sizes.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 81 of its 99 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

81 of 99 references.

Closest ancestors · 21

Same direction, newest first

9 more

Other references · 60

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2020Denoising Diffusion Probabilistic Modelscited by 20,851
2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
2023GPT-4 Technical Reportcited by 18,132
2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
2020Denoising Diffusion Implicit Modelscited by 9,655
2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
2023Visual Instruction Tuningcited by 7,792
2017Mask R-CNNcited by 7,259
2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986

38 more

2022Classifier-Free Diffusion Guidancecited by 5,717
2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
2021Image Super-Resolution via Iterative Refinementcited by 1,506
2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
2018Demystifying MMD GANscited by 1,268
2022Denoising Diffusion Restoration Modelscited by 951
2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
2021Multi-Stage Progressive Image Restorationcited by 813
2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
2018Deep Retinex Decomposition for Low-Light Enhancementcited by 720
2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 574
2023Zero-shot Image-to-Image Translationcited by 503
2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 486
2017Benchmarking Single Image Dehazing and Beyondcited by 445
2016Deep Joint Rain Detection and Removal from a Single Imagecited by 351
2021Towards Real-World Blind Face Restoration with Generative Facial Priorcited by 347
2019Multi-Label Image Recognition with Graph Convolutional Networkscited by 320
2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 255
2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 253
2021Deblurring via Stochastic Refinementcited by 248
2022Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Modelscited by 215
2017DesnowNet: Context-Aware Deep Network for Snow Removalcited by 200
2023Generative Diffusion Prior for Unified Image Restoration and Enhancementcited by 198
2019Spatial Attentive Single-Image Deraining with a High Quality Real Rain Datasetcited by 192
2023Low-Light Image Enhancement with Wavelet-based Diffusion Modelscited by 126
2023Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Modelcited by 118
2022VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decodercited by 116
2023PromptIR: Prompting for All-in-One Blind Image Restorationcited by 113
2022ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removalcited by 102
2023Implicit Neural Representation for Cooperative Low-light Image Enhancementcited by 92

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 20; 2026: 16.

47 papers.

Sort:

By area:

2024PromptFix: You Prompt and We Fix the Photocited by 40
2025Visual-Instructed Degradation Diffusion for All-in-One Image Restorationcited by 26
2024Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approachcited by 24
2025UltraFusion: Ultra High Dynamic Imaging using Exposure Fusioncited by 20
2024AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancementcited by 16
2024Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restorationcited by 15
2025RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restorationcited by 15
2024GenDeg: Diffusion-based Degradation Synthesis for Generalizable All-In-One Image Restorationcited by 13
2024WeatherGS: 3D Scene Reconstruction in Adverse Weather Conditions via Gaussian Splattingcited by 13
2025Exploiting Diffusion Prior for Real-World Image Dehazing with Unpaired Trainingcited by 12
2024Boosting Weakly-Supervised Referring Image Segmentation via Progressive Comprehensioncited by 10
2025Reversing Flow for Image Restorationcited by 10

35 more

2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxycited by 8
2025When Schr"odinger Bridge Meets Real-World Image Dehazing with Unpaired Trainingcited by 8
2025Exploiting Diffusion Prior for Task-driven Image Restorationcited by 7
2025UniSER: A Foundation Model for Unified Soft Effects Removalcited by 5
2025BlurDM: A Blur Diffusion Model for Image Deblurringcited by 5
2025Uni-DocDiff: A Unified Document Restoration Model Based on Diffusioncited by 4
2025FideDiff: Efficient Diffusion Model for High-Fidelity Image Motion Deblurringcited by 4
2024ReviveDiff: A Universal Diffusion Model for Restoring Images in Adverse Weather Conditionscited by 3
2024Zero-Shot Low Light Image Enhancement with Diffusion Priorcited by 3
2025TDM: Temporally-Consistent Diffusion Model for All-in-One Real-World Video Restorationcited by 3
2025Latent Harmony: Synergistic Unified UHD Image Restoration via Latent Space Regularization and Controllable Refinementcited by 3
2026TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restorationcited by 3
2024Data-free Distillation with Degradation-prompt Diffusion for Multi-weather Image Restorationcited by 2
2025Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancementcited by 2
2025Zero-Shot Video Deraining with Video Diffusion Modelscited by 2
2026NanoSD: Edge Efficient Foundation Model for Real Time Image Restorationcited by 2
2026NimbusGS: Unified 3D Scene Reconstruction under Hybrid Weathercited by 2
2024Consistent Diffusion: Denoising Diffusion Model with Data-Consistent Training for Image Restorationcited by 1
2025Physics-Guided Image Dehazing Diffusioncited by 1
2025Moir'eXNet: Adaptive Multi-Scale Demoir'eing with Linear Attention Test-Time Training and Truncated Flow Matching Priorcited by 1
2026HSI-VAR: Rethinking Hyperspectral Restoration through Spatial-Spectral Visual Autoregressioncited by 1
2026HNDiff: Haze-Noise Diffusion for Image Dehazingcited by 1
2025Pathology Image Restoration via Mixture of Promptscited by 0
2025Seeing the Unseen in Low-light Spike Streamscited by 0
2026Toward Generalizable Deblurring: Leveraging Massive Blur Priors with Linear Attention for Real-World Scenarioscited by 0
2026Seeing Through the PRISM: Compound & Controllable Restoration of Scientific Imagescited by 0
2026Your Pre-trained Diffusion Model Secretly Knows Restorationcited by 0
2026A Noise Constrained Diffusion (NC-Diffusion) Framework for High Fidelity Image Compressioncited by 0
2026Beyond Fidelity: Semantic Similarity Assessment in Low-Level Image Processingcited by 0
2026R2H-Diff: Guided Spectral Diffusion Model for RGB-to-Hyperspectral Reconstructioncited by 0
2026EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learningcited by 0
2026Geodesic Flow Matching on a Riemannian Degradation Manifold for Blind Image Restorationcited by 0
2026Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restorationcited by 0
2026ImIR: Image-Instruction Tuning for All-in-One Image Restorationcited by 0
2026PickMoment: Continuous-Time Single-Image-to-Video via Learning Deblurring and Blur-to-Videocited by 0

Similar, not cited

Close in wording, no citation either way