# Token Perturbation Guidance for Diffusion Models

arXiv: https://arxiv.org/abs/2506.10036 · HTML: https://mlsanity.com/paper/2506.10036/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 33 of its 37 references that are in the map.
- First posted on arXiv 10 Jun 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 33)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Lumiere: A Space-Time Diffusion Model for Video Generation](https://mlsanity.com/paper/2401.12945/) (2024)
- [Guiding a Diffusion Model with a Bad Version of Itself](https://mlsanity.com/paper/2406.02507/) (2024)
- [Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models](https://mlsanity.com/paper/2404.07724/) (2024)
- [Improving Sample Quality of Diffusion Models Using Self-Attention Guidance](https://mlsanity.com/paper/2210.00939/) (2022)
- [CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling](https://mlsanity.com/paper/2310.17347/) (2023)
- [Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models](https://mlsanity.com/paper/2410.02416/) (2024)
- [Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance](https://mlsanity.com/paper/2403.17377/) (2024)
- [Stable Flow: Vital Layers for Training-Free Image Editing](https://mlsanity.com/paper/2411.14430/) (2024)
- [Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention](https://mlsanity.com/paper/2408.00760/) (2024)
- [Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later](https://mlsanity.com/paper/2303.02490/) (2023)
- [Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation](https://mlsanity.com/paper/2401.01044/) (2024)
- [No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Models](https://mlsanity.com/paper/2407.02687/) (2024)
- [VideoTetris: Towards Compositional Text-to-Video Generation](https://mlsanity.com/paper/2406.04277/) (2024)
- [Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition](https://mlsanity.com/paper/2403.14148/) (2024)
- [AudioX: A Unified Framework for Anything-to-Audio Generation](https://mlsanity.com/paper/2503.10522/) (2025)

## Built on by (most cited first, 15 of 15)

- [UniGame: Turning a Unified Multimodal Model Into Its Own Adversary](https://mlsanity.com/paper/2511.19413/) (2025)
- [Unleashing Guidance Without Classifiers for Human-Object Interaction Animation](https://mlsanity.com/paper/2603.25734/) (2026)
- [PickStyle: Video-to-Video Style Transfer with Context-Style Adapters](https://mlsanity.com/paper/2510.07546/) (2025)
- [Guiding a Diffusion Model by Swapping Its Tokens](https://mlsanity.com/paper/2604.08048/) (2026)
- [STRIDE: Training-Free Diversity Guidance via PCA-Directed Feature Perturbation in Single-Step Diffusion Models](https://mlsanity.com/paper/2605.11494/) (2026)
- [SoftCFG: Uncertainty-guided Stable Guidance for Visual Autoregressive Model](https://mlsanity.com/paper/2510.00996/) (2025)
- [Improving Diffusion Generalization with Weak-to-Strong Segmented Guidance](https://mlsanity.com/paper/2603.20584/) (2026)
- [Prompt-aware classifier free guidance for diffusion models](https://mlsanity.com/paper/2509.22728/) (2025)
- [TAG: Tangential Amplifying Guidance for Hallucination-Resistant Sampling](https://mlsanity.com/paper/2510.04533/) (2025)
- [Guiding Visual Autoregressive Models through Spectrum Weakening](https://mlsanity.com/paper/2511.22991/) (2025)
- [Geometry-Aware Attention Guidance for Diffusion Models via Modern Hopfield Dynamics](https://mlsanity.com/paper/2603.02531/) (2026)
- [Guiding Diffusion Models with Semantically Degraded Conditions](https://mlsanity.com/paper/2603.10780/) (2026)
- [ABOPD: Antibody CDR Design via On-Policy Distillation](https://mlsanity.com/paper/2607.18835/) (2026)
- [Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation](https://mlsanity.com/paper/2608.14172/) (2026)
- [Opening the Black Box of Classifier-Free Guidance via Information Bottleneck](https://mlsanity.com/paper/2610.13369/) (2026)

## Abstract (arXiv)

Classifier-free guidance (CFG) has become an essential component of modern diffusion models to enhance both generation quality and alignment with input conditions. However, CFG requires specific training procedures and is limited to conditional generation. To address these limitations, we propose Token Perturbation Guidance (TPG), a novel method that applies perturbation matrices directly to intermediate token representations within the diffusion network. TPG employs a norm-preserving shuffling operation to provide effective and stable guidance signals that improve generation quality without architectural changes. As a result, TPG is training-free and agnostic to input conditions, making it readily applicable to both conditional and unconditional generation. We further analyze the guidance term provided by TPG and show that its effect on sampling more closely resembles CFG compared to existing training-free guidance techniques. Extensive experiments on SDXL and Stable Diffusion 2.1 show that TPG achieves nearly a 2× improvement in FID for unconditional generation over the SDXL baseline, while closely matching CFG in prompt alignment. These results establish TPG as a general, condition-agnostic guidance method that brings CFG-like benefits to a broader class of diffusion models.