Dynamic VLM-Guided Negative Prompting for Diffusion Models
On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment
also in 1 more
Abstract (arXiv)
We propose a novel approach for dynamic negative prompting in diffusion models that leverages Vision-Language Models (VLMs) to adaptively generate negative prompts during the denoising process. Unlike traditional Negative Prompting methods that use fixed negative prompts, our method generates intermediate image predictions at specific denoising steps and queries a VLM to produce contextually appropriate negative prompts. We evaluate our approach on various benchmark datasets and demonstrate the trade-offs between negative guidance strength and text-image alignment.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 18 references that are in the map. First posted on arXiv 30 Oct 2025.
Builds on
16 of 18 references.
References · 16
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
- 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 239
- 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
4 more
- 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 161
- 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
- 2024Understanding the Impact of Negative Prompts: When and How Do They Take Effect?cited by 50
- 2024Dynamic Negative Guidance of Diffusion Modelscited by 28
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Diffusion Models with Adaptive Negative Sampling Without External Resources
- 2024Improving image synthesis with diffusion-negative sampling
- 2025Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process
- 2026Advanced Image Generation: Negative Prompt Optimization and Latent Classifier Guidance
- 2026PAST: Prompt-Adaptive Sampling Termination for Efficient Diffusion Model
- 2025Noise Projection: Closing the Prompt-Agnostic Gap Behind Text-to-Image Misalignment in Diffusion Models
- 2025Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervision
- 2024On Discrete Prompt Optimization for Diffusion Models