mlsanityData to 14 Oct 2026

Token Perturbation Guidance for Diffusion Models

arXiv 2506.10036 ↗First posted 10 Jun 2025cs.GR cs.CLcited by 15

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Classifier-free guidance (CFG) has become an essential component of modern diffusion models to enhance both generation quality and alignment with input conditions. However, CFG requires specific training procedures and is limited to conditional generation. To address these limitations, we propose Token Perturbation Guidance (TPG), a novel method that applies perturbation matrices directly to intermediate token representations within the diffusion network. TPG employs a norm-preserving shuffling operation to provide effective and stable guidance signals that improve generation quality without architectural changes. As a result, TPG is training-free and agnostic to input conditions, making it readily applicable to both conditional and unconditional generation. We further analyze the guidance term provided by TPG and show that its effect on sampling more closely resembles CFG compared to existing training-free guidance techniques. Extensive experiments on SDXL and Stable Diffusion 2.1 show that TPG achieves nearly a 2× improvement in FID for unconditional generation over the SDXL baseline, while closely matching CFG in prompt alignment. These results establish TPG as a general, condition-agnostic guidance method that brings CFG-like benefits to a broader class of diffusion models.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 37 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

33 of 37 references.

Closest ancestors · 17

Same direction, newest first

  1. 2024Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models3 Oct
  2. 2024Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention1 Aug
  3. 2024No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Models2 Jul
  4. 2024Guiding a Diffusion Model with a Bad Version of Itself4 Jun
  5. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models11 Apr
  6. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance26 Mar
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  8. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling26 Oct
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  10. 2023Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later4 Mar
  11. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidance3 Oct
  12. 2022Classifier-Free Diffusion Guidance26 Jul
5 more
  1. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  2. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  4. 2020Denoising Diffusion Implicit Models6 Oct
  5. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  8. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  9. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 113
  10. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generationcited by 54
  11. 2024VideoTetris: Towards Compositional Text-to-Video Generationcited by 51
  12. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 42
4 more
  1. 2025AudioX: A Unified Framework for Anything-to-Audio Generationcited by 27
  2. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusioncited by 20
  3. 2024NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generationcited by 13
  4. 2024DreamDissector: Learning Disentangled Text-to-3D Generation from 2D Diffusion Priorscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

By area:

  1. 2025UniGame: Turning a Unified Multimodal Model Into Its Own Adversarycited by 9
  2. 2026Unleashing Guidance Without Classifiers for Human-Object Interaction Animationcited by 6
  3. 2025PickStyle: Video-to-Video Style Transfer with Context-Style Adapterscited by 3
  4. 2026Guiding a Diffusion Model by Swapping Its Tokenscited by 3
  5. 2026STRIDE: Training-Free Diversity Guidance via PCA-Directed Feature Perturbation in Single-Step Diffusion Modelscited by 3
  6. 2025SoftCFG: Uncertainty-guided Stable Guidance for Visual Autoregressive Modelcited by 2
  7. 2026Improving Diffusion Generalization with Weak-to-Strong Segmented Guidancecited by 1
  8. 2025Prompt-aware classifier free guidance for diffusion modelscited by 0
  9. 2025TAG: Tangential Amplifying Guidance for Hallucination-Resistant Samplingcited by 0
  10. 2025Guiding Visual Autoregressive Models through Spectrum Weakeningcited by 0
  11. 2026Geometry-Aware Attention Guidance for Diffusion Models via Modern Hopfield Dynamicscited by 0
  12. 2026Guiding Diffusion Models with Semantically Degraded Conditionscited by 0
3 more
  1. 2026ABOPD: Antibody CDR Design via On-Policy Distillationcited by 0
  2. 2026Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generationcited by 0
  3. 2026Opening the Black Box of Classifier-Free Guidance via Information Bottleneckcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Domain Guidance: A Simple Transfer Approach for a Pre-trained Diffusion Model
  2. 2024TFG: Unified Training-Free Guidance for Diffusion Models
  3. 2023Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance
  4. 2024Rectified Diffusion Guidance for Conditional Generation
  5. 2025Improving Classifier-Free Guidance in Masked Diffusion: Low-Dim Theoretical Insights with High-Dim Impact
  6. 2025Feedback Guidance of Diffusion Models
  7. 2026Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models
  8. 2025Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Models