# Null-text Guidance in Diffusion Models is Secretly a Cartoon-style Creator

arXiv: https://arxiv.org/abs/2305.06710 · HTML: https://mlsanity.com/paper/2305.06710/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 36 of its 45 references that are in the map.
- First posted on arXiv 11 May 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 36)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality](https://mlsanity.com/paper/2202.05830/) (2022)
- [Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models](https://mlsanity.com/paper/2207.13038/) (2022)
- [A Style-Aware Content Loss for Real-time HD Style Transfer](https://mlsanity.com/paper/1807.10201/) (2018)
- [3DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models](https://mlsanity.com/paper/2211.14108/) (2022)
- [Denoising Likelihood Score Matching for Conditional Score-based Data Generation](https://mlsanity.com/paper/2203.14206/) (2022)

## Built on by (most cited first, 12 of 12)

- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Diffusion Models as Cartoonists: The Curious Case of High Density Regions](https://mlsanity.com/paper/2411.01293/) (2024)
- [Controlling Human Shape and Pose in Text-to-Image Diffusion Models via Domain Adaptation](https://mlsanity.com/paper/2411.04724/) (2024)
- [Cartoondiff: Training-free Cartoon Image Generation with Diffusion Transformer Models](https://mlsanity.com/paper/2309.08251/) (2023)
- [ColorEdit: Training-free Image-Guided Color editing with diffusion model](https://mlsanity.com/paper/2411.10232/) (2024)
- [Diffusion Models are Secretly Zero-Shot 3DGS Harmonizers](https://mlsanity.com/paper/2503.06740/) (2025)
- [Advancing Facial Stylization through Semantic Preservation Constraint and Pseudo-Paired Supervision](https://mlsanity.com/paper/2506.22022/) (2025)
- [CEIDM: A Controlled Entity and Interaction Diffusion Model for Enhanced Text-to-Image Generation](https://mlsanity.com/paper/2508.17760/) (2025)
- [MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Models](https://mlsanity.com/paper/2510.17519/) (2025)
- [BeyondFacial: Identity-Preserving Personalized Generation Beyond Facial Close-ups](https://mlsanity.com/paper/2511.11989/) (2025)
- [Wavelet-Guided Semantic Signal Compensation for Inversion-Free Image Editing](https://mlsanity.com/paper/2607.02421/) (2026)

## Abstract (arXiv)

Classifier-free guidance is an effective sampling technique in diffusion models that has been widely adopted. The main idea is to extrapolate the model in the direction of text guidance and away from null-text guidance. In this paper, we demonstrate that null-text guidance in diffusion models is secretly a cartoon-style creator, i.e., the generated images can be efficiently transformed into cartoons by simply perturbing the null-text guidance. Specifically, we proposed two disturbance methods, i.e., Rollback disturbance (Back-D) and Image disturbance (Image-D), to construct misalignment between the noisy images used for predicting null-text guidance and text guidance (subsequently referred to as null-text noisy image and text noisy image respectively) in the sampling process. Back-D achieves cartoonization by altering the noise level of null-text noisy image via replacing xt with xt+Delta t. Image-D, alternatively, produces high-fidelity, diverse cartoons by defining xt as a clean input image, which further improves the incorporation of finer image details. Through comprehensive experiments, we delved into the principle of noise disturbing for null-text and uncovered that the efficacy of disturbance depends on the correlation between the null-text noisy image and the source image. Moreover, our proposed techniques, which can generate cartoon images and cartoonize specific ones, are training-free and easily integrated as a plug-and-play component in any classifier-free guided diffusion model. Project page is available at urlhttps://nulltextforcartoon.github.io/.